AI News

Trending story

METR Uses Investigative Agents to Measure Identification Cues in Alignment Evaluations

1 reports1 story updatesUpdated Nov 16 16:00

Understand this story

AI overview

On November 16, 2026, METR released a research note in which investigative agents were run separately in 19 alignment evaluation environments and 36 capability evaluation environments to measure their ability to judge the nature of the environments. The results showed that Opus 5.5's direct scores and open-ended scores ranked every alignment evaluation above all capability evaluations on the dataset, with AUC close to perfect; the agents mainly made judgments based on cues such as the environment appearing fake and the task resembling a honeypot. The study divided alignment cues into two categories: environment-level (simulation not realistic) and task-level (designed to induce undesirable behavior), and proposed that investigative agents can be used to iteratively improve the realism of alignment evaluations.

AI-generated from public reports

Latest development · Report

METR study: How investigative agents identify alignment evaluations

Developments

1 confirmed developments

  1. METR uses survey agents to measure identification cues in alignment evaluations

    Source:METR Notes

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 16

  1. METR NotesFirst-party

    METR study: How investigative agents identify alignment evaluations

    METR published a research note in which investigative agents were placed into 19 alignment evaluation environments and 36 capability evaluation environments to measure their ability to judge the nature of the environment. The results showed that Opus 5.5's direct scoring and open-ended scoring ranked every alignment evaluation above all capability evaluations on the dataset, with an AUC close to perfect; the agents made their judgments mainly based on cues such as the environment appearing fake and the task resembling a honeypot. The study classifies alignment cues into two types: environment-level (simulation not realistic) and task-level (designed to elicit undesirable behavior), and proposes that investigative agents can be used to iteratively improve the realism of alignment evaluations.

Related stories

  1. Related storiesBerkeley RDI proposes adversarial cybersecurity evaluation benchmark CyberArena
  2. Related storiesAnthropic releases MAMUBench, a multi-user multi-agent benchmark
  3. Related storiesAnthropic launches Claude Fable 5.2 and Claude Mythos 5.2
  4. Related storiesOpenAI launches GPT-6.1 Terra on ChatGPT Work and Codex