The Northwestern and Google paper proposes Harness2, enabling agent harnesses to self-improve without answer keys
A new paper from Northwestern and Google proposes Harness2, which enables agent harnesses to self-improve on new tasks without answer keys: first run the agent, have an editor model propose harness modifications, then rerun the tasks to see which modifications actually improved behavior, and store the experience from each task in a library to guide the next modification. One improvement step raised GLM-5's WorkBuddy-Bench score by 11.5 points, surpassing Gemini 3.5 Flash using the original harness; whereas modifying the harness directly without these test runs dropped the coding score from 73.4 to 63.2.
Why read this
AI score 58
The original text presents the specific process for automatically rewriting the agent harness in scenarios without answer keys, along with the score changes brought by one improvement and a counterexample.
Topics
Body
New Northwestern and Google paper shows that agent harnesses can improve themselves on new tasks without answer keys, by test-running edits and keeping what helps.
An agent's harness is the prompts, tools, skills, and memory wrapped around the model. Tuning it by hand is slow, and new real-world tasks come with no answer key to tune against.
Harness2 runs the agent, has an editor model propose harness fixes, then reruns the task to see which fixes improved the agent's behavior. Lessons from each task go into a library that guides edits on the next one.
A single improvement step raised GLM-5's WorkBuddy-Bench score by 11.5 points, past Gemini 3.5 Flash on its stock harness. Editing the harness without those test runs instead dropped coding scores from 73.4 to 63.2.
Treat your agent's harness as something to revise per task, and keep only the edits that change its behavior on real runs.
AI Trends