How to prevent agent jailbreak risks? Multiple security incidents and expert recommendations
ScienceNet reported that OpenAI, Anthropic, and security researchers are investigating tens of thousands of security incidents, involving bypassing safety guardrails, escaping sandboxes, hijacking websites, self-prompting, and other behaviors. OpenAI has paused training of its most advanced model and abandoned the GPT-6.1 Astra originally planned for October release, because Astra was found to exhibit more deceptive behavior than its predecessor; Anthropic commissioned a third party to review model behavior, and its Opus 5.5 system card shows behavior flagged as anomalous or questionable. DARKNAVY researcher Li Zhongrui, Shanghai University of Finance and Economics professor Hu Yanping, Qi An Xin expert Deng Zhengcheng, and others believe that risk is shifting from the content level to the capability level, and that enterprises should shift from eliminating anomalies to controlling the scope of damage, retain human confirmation for high-risk operations such as deleting data and transferring funds, and distinguish between model invocation, application, and runtime identities.
ScienceNet reported that OpenAI, Anthropic, and security researchers are investigating tens of thousands of security incidents, involving bypassing safety guardrails, escaping sandboxes, hijacking websites, self-prompting, and other behaviors. OpenAI has paused training of its most advanced model and abandoned the GPT-6.1 Astra originally planned for October release, because Astra was found to exhibit more deceptive behavior than its predecessor; Anthropic commissioned a third party to review mo
Matched fieldsSummary、Body