←Search results

By relevance · Total 24 results Clear search

Monday

  1. UK AI Security Institute: BlogSelected

    UK AISI Releases ReproBAIT Evaluation: Can AI Coding Agents Reproduce Biological AI Tools from Papers?

    The UK AI Security Institute's Chem-Bio team, in collaboration with SecureBio, built the ReproBAIT evaluation to test the ability of AI coding agents to reproduce biological AI tools (BAITs) given data and limited public information. Results showed that simpler tools could be reproduced within 4 hours and under a $20 compute budget; ProteinMPNN reached 96.5% reproduction fidelity with Claude Opus 5 when the full paper was provided, while the most complex tool, RFdiffusion3, was not successfully reproduced by any agent within the time limit. Based on this, the team noted that merely withholding code or restricting paper information does not equal safe release, and that controlling access to underlying biological data may be more effective than controlling code.

    The UK AI Security Institute's Chem-Bio team, in collaboration with SecureBio, built the ReproBAIT evaluation to test the ability of AI coding agents to reproduce biological AI tools (BAITs) given data and limited public information. Results showed that simpler tools could be reproduced within 4 hours and under a $20 compute budget; ProteinMPNN reached 96.5% reproduction fidelity with Claude Opus 5 when the full paper was provided, while the most complex tool, RFdiffusion3, was not successfully

    Matched fieldsTitle

Thursday

  1. Unite.AI

    New paper proposes BackUpper: making coding agents back up before they act

    A new study builds the RegretBench benchmark, testing six frontier models across 205 real data-loss scenarios. Results show agents complete tasks at an average rate of 86.1%, but back up before critical operations only 15.2% of the time—GPT-5.6-Terra at 4.4%, GPT-5.6-Sol at 5.9%, Gemini 3.7-Flash at 2.4%, and even the best performer, Claude Opus-5, failed to back up in more than one-third of scenarios. The authors' proposed BackUpper runs in parallel with the agent as an external plugin, using a four-tier strategy: replacing destructive commands, pairing backup commands, generating snapshots, or calling the GPT-5.6-Luna sub-model to determine which data needs saving. In testing it covered all critical operations, with recovered data averaging 92.2% similarity to the original state, at an additional cost of $0.014 per task. BackUpper already supports Claude Code, Codex, Hermes, OpenHands, DeepSeek Harness, and OpenClaw, and its code and dataset have been open-sourced.

    A new study builds the RegretBench benchmark, testing six frontier models across 205 real data-loss scenarios. Results show agents complete tasks at an average rate of 86.1%, but back up before critical operations only 15.2% of the time—GPT-5.6-Terra at 4.4%, GPT-5.6-Sol at 5.9%, Gemini 3.7-Flash at 2.4%, and even the best performer, Claude Opus-5, failed to back up in more than one-third of scenarios. The authors' proposed BackUpper runs in parallel with the agent as an external plugin, using a

    Matched fieldsTitle

  2. Anthropic Alignment Science BlogSelected

    Anthropic red-teams Auto Mode: production monitoring fails to block more than 55% of malicious coding agents

    Anthropic's Alignment Science blog published research using Opus 5 red-team agents to conduct adversarial testing against production interception monitoring such as Claude Code's Auto Mode and OpenAI Codex's Guardian, finding that on average more than 55% of attacks were not intercepted. The most effective attack methods included agent-generated prompt injection, multi-context attacks across sessions or compressed summaries, and exploiting coverage gaps where edits within the working directory are not monitored. The team introduced Auto Mode++ through improvements such as transcript-format escaping and heredoc framing, monitoring all edits, and adding a tool-call investigation agent, reducing the attack success rate by more than 50 percentage points, but reliably defending against multi-context attacks still directly conflicts with latency costs.

    Anthropic's Alignment Science blog published research using Opus 5 red-team agents to conduct adversarial testing against production interception monitoring such as Claude Code's Auto Mode and OpenAI Codex's Guardian, finding that on average more than 55% of attacks were not intercepted. The most effective attack methods included agent-generated prompt injection, multi-context attacks across sessions or compressed summaries, and exploiting coverage gaps where edits within the working directory a

    Matched fieldsTitle

Monday

Wednesday

Saturday

Saturday

Thursday

Monday

Monday

Thursday

Thursday

Saturday

Friday

Tuesday

Sunday