Understand this story
The UK AI Security Institute's Chem-Bio team, in collaboration with SecureBio, built the ReproBAIT (Reproducing Biological AI Tools) evaluation to test the ability of AI coding agents to reproduce biological AI tools given data and limited public information. The results showed that simpler tools could be reproduced within 4 hours and under a $20 compute budget; ProteinMPNN reached 96.5% reproduction fidelity with Claude Opus 5 when the full paper was provided; and for the most complex RFdiffusion3, no agent succeeded in reproducing it within the time limit. The team therefore noted that releasing only code or restricting paper information does not equal safe release, and that controlling access to the underlying biological data may be more effective than controlling code.
AI-generated from public reports
Latest development · Report
UK AISI Releases ReproBAIT Evaluation: Can AI Coding Agents Reproduce Biological AI Tools from Papers?Developments
1 confirmed developments
- Source:English text is being prepared.
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 26
UK AI Security Institute: BlogFirst-partySelected
UK AISI Releases ReproBAIT Evaluation: Can AI Coding Agents Reproduce Biological AI Tools from Papers?
The UK AI Security Institute's Chem-Bio team, in collaboration with SecureBio, built the ReproBAIT evaluation to test the ability of AI coding agents to reproduce biological AI tools (BAITs) given data and limited public information. Results showed that simpler tools could be reproduced within 4 hours and under a $20 compute budget; ProteinMPNN reached 96.5% reproduction fidelity with Claude Opus 5 when the full paper was provided, while the most complex tool, RFdiffusion3, was not successfully reproduced by any agent within the time limit. Based on this, the team noted that merely withholding code or restricting paper information does not equal safe release, and that controlling access to underlying biological data may be more effective than controlling code.