AI Trends

Trending story

Northwestern and Google paper proposes a self-improving agent harness

1 reports1 story updatesUpdated Nov 15 08:08

Understand this story

AI overview

Northwestern and Google published a paper proposing Harness2, which enables an agent harness to self-improve on new tasks without answer keys. Its process is to first run the agent, have an editor model propose harness modifications, then rerun the tasks to verify which modifications genuinely improve behavior, and store the experience from each task in a library to guide the next modification. The paper reports that a single improvement step raised GLM-5's WorkBuddy-Bench score by 11.5 points, surpassing Gemini 3.5 Flash using the original harness; whereas modifying the harness directly without test runs dropped the coding score from 73.4 to 63.2.

AI-generated from public reports

Latest development · Report

The Northwestern and Google paper proposes Harness2, enabling agent harnesses to self-improve without answer keys

Developments

1 confirmed developments

  1. Northwestern and Google paper proposes a self-improving agent harness

    Source:Rohan Paul

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 15

  1. Rohan Paul

    The Northwestern and Google paper proposes Harness2, enabling agent harnesses to self-improve without answer keys

    A new paper from Northwestern and Google proposes Harness2, which enables agent harnesses to self-improve on new tasks without answer keys: first run the agent, have an editor model propose harness modifications, then rerun the tasks to see which modifications actually improved behavior, and store the experience from each task in a library to guide the next modification. One improvement step raised GLM-5's WorkBuddy-Bench score by 11.5 points, surpassing Gemini 3.5 Flash using the original harness; whereas modifying the harness directly without these test runs dropped the coding score from 73.4 to 63.2.

Related stories

  1. Related storiesMIT and Google DeepMind paper finds reporting-line bias in AI advisors
  2. Related storiesDeepMind and Columbia propose BTTF agent-based chip simulation debugging method
  3. Related storiesGoogle DeepMind paper shows prompt injection remains unsolved
  4. Related storiesGoogle DeepMind and UChicago paper finds that AI agents' speaking style affects group policy