AI Trends

Trending story

Redwood Research Conducts an Experiment on Detecting Sabotage in AI Safety Research

1 reports1 story updatesUpdated Nov 21 07:28

Understand this story

AI overview

Redwood Research had human experts secretly sabotage AI safety research and asked auditors to inspect the complete research logs. The experiment covered 22 research records and 33 specific sabotage mechanisms, with 25 human auditors completing 104 audits. Human auditing achieved an AUROC of 0.59, fully identified 17% of the sabotage mechanisms, and completely missed 74%. Claude Fable 5.1 and GPT-6 Astra scored 37.1% and 32.6%, respectively, on mechanism identification—higher than the human score of 21.9%—but still missed most of the sabotage. Redwood Research also noted that all samples came from a single researcher, the confidence intervals were very wide, and the saboteurs knew the normal experimental results in advance. There are currently no other evidence-backed developments.

AI-generated from public reports

Latest development · Report

Redwood Research: Auditors Struggle to Detect Deliberate Sabotage in Full AI Safety Research Records

Developments

1 confirmed developments

  1. Redwood Research Conducts an Experiment on Detecting Sabotage in AI Safety Research

    Source:Redwood Research:Blog

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 21

  1. Redwood Research:BlogFirst-partySelected

    Redwood Research: Auditors Struggle to Detect Deliberate Sabotage in Full AI Safety Research Records

    Redwood Research had human experts secretly sabotage AI safety research and then asked auditors to examine the complete research logs. Human auditors achieved an AUROC of just 0.59, fully identified 17% of the sabotage mechanisms, and completely missed 74%. The experiment included 22 research records and 33 specific sabotage mechanisms, with 25 human auditors completing 104 audits. Claude Fable 5.1 and GPT-6 Astra scored 37.1% and 32.6%, respectively, on mechanism identification—higher than the human score of 21.9%—but still missed most of the sabotage. The study also noted that all samples came from a single researcher, the confidence intervals were very wide, and the saboteurs knew the normal experimental results in advance.

Related stories

  1. Related storiesThinky Machines Releases Inkling 2 Mini
  2. Related storiesOpenAI Shares Research on Countering Model Distillation and Reasoning Extraction
  3. Related storiesAnthropic launches Claude Fable 5.2 and Claude Mythos 5.2
  4. Related storiesApple Research Proposes Information-Theoretic Limits of KL-Regularized Alignment