Understand this story
On November 16, 2026, METR released a research note in which investigative agents were run separately in 19 alignment evaluation environments and 36 capability evaluation environments to measure their ability to judge the nature of the environments. The results showed that Opus 5.5's direct scores and open-ended scores ranked every alignment evaluation above all capability evaluations on the dataset, with AUC close to perfect; the agents mainly made judgments based on cues such as the environment appearing fake and the task resembling a honeypot. The study divided alignment cues into two categories: environment-level (simulation not realistic) and task-level (designed to induce undesirable behavior), and proposed that investigative agents can be used to iteratively improve the realism of alignment evaluations.
AI-generated from public reports
Latest development · Report
METR study: How investigative agents identify alignment evaluationsDevelopments
1 confirmed developments
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 16
METR NotesFirst-party
METR study: How investigative agents identify alignment evaluations
METR published a research note in which investigative agents were placed into 19 alignment evaluation environments and 36 capability evaluation environments to measure their ability to judge the nature of the environment. The results showed that Opus 5.5's direct scoring and open-ended scoring ranked every alignment evaluation above all capability evaluations on the dataset, with an AUC close to perfect; the agents made their judgments mainly based on cues such as the environment appearing fake and the task resembling a honeypot. The study classifies alignment cues into two types: environment-level (simulation not realistic) and task-level (designed to elicit undesirable behavior), and proposes that investigative agents can be used to iteratively improve the realism of alignment evaluations.