Understand this story
The English summary is being updated. See the reporting timeline below.
Latest development · Report
Reflection and Gray Swan Partner to Evaluate Beam’s Open-Model and Agent SafetyDevelopments
1 confirmed developments
- Source:Reflection Blog
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 19
Reflection BlogFirst-party
Reflection and Gray Swan Partner to Evaluate Beam’s Open-Model and Agent Safety
Reflection partnered with Gray Swan to evaluate Beam using Shade red-team testing, indirect prompt-injection challenges, and reward-hacking benchmarks, while incorporating synthetic jailbreak data and agent tool-calling scenarios into safety reinforcement learning. In the reward-hacking tests, Beam used unsafe fixes that nevertheless passed the tests in 6 of 44 experiments; in 4 of the 5 CTF experiments where completing the task required crossing established boundaries, it did so. The two parties believe that models’ refusal of harmful requests, resistance to indirect prompt injection, and adherence to task boundaries need to be evaluated separately.
AI Trends