AI Trends

Google and Penn State University paper proposes SIM, recording rationales for agent framework changes

Rohan PaulUpdated Nov 18 05:02

AI summary

A new paper from Google and Penn State University proposes Self-Improvement Memory (SIM), which, when one AI rewrites another AI's instructions and tool code, records the problem each change targets, the reasoning process, and the measured results. When transferring Claude Opus 5.5's harness to Gemini 3.8 Flash's workspace tasks, SIM raised the score from 68.41 to 84.75, versus 73.42 without logs; across 10 repeated coding runs, SIM led in 9 of them, averaging 2.36 points higher.

Why read this

AI score 57

The paper presents an approach that preserves change rationales during cross-model transfer, and uses a set of concrete score comparisons to illustrate the difference logging makes.

Topics

Tags

Body

New Google and Penn State paper finds that when one AI rewrites the code that gives another AI its instructions and tools, keeping notes on why each change was made helps the gains carry over to a new model.

LLMs can now rewrite an agent's harness, the code that runs prompts and tools. After a model or benchmark switch, it can score below the original, and its code hides why edits were made.

Self-Improvement Memory, or SIM, logs each edit with the problem it targeted, its reasoning and its measured result. The optimizer reads this log on the new setup to decide which old edits to keep or revise.

When a Claude Opus 5.5 harness moved to Gemini 3.8 Flash workspace tasks, SIM lifted it from 68.41 to 84.75, versus 73.42 without the log. Across 10 repeated coding runs, SIM finished ahead 9 times, by 2.36 points on average.

Store why each harness edit was made and how it scored, and carry that log to new models.

Source

Rohan Paul