Import AI 468: 23 RSI Policy Recommendations, PostTrainBench+, and Trust and Transparency in the AI Race

Want to be able to deal with RSI? Here are 23 actionable policy ideas: …IFP serves up some “low-regret” policy recommendations… Policy experts with think tank IFP have published a set of ideas meant to help “policymakers begin addressing the risks of further automating AI R&D”. The recommendations involve 23 specific ideas falling across 7 specific categories. If adopted, these recommendations would also give countries, especially the United States, more moves they can make on the gameboard as powerful systems are developed, ideally giving them the ability to:

Accelerate “the diffusion of AI capabilities, by allocating compute and talent towards inference and the development of new AI applications”.

Accelerate “R&D to make further AI research automation safer, either by improving model safety directly or by boosting societal resilience”.

Seven categories of idea:

“Provide transparency into automated AI R&D

Improve state capacity to understand and respond to automated AI R&D

Develop a risk management strategy for automated AI R&D that accelerates defensive and commercial AI uses

Accelerate the development of AI verification technology

Invest in AI resilience

Extend the US AI lead to give the US more time to manage AI R&D automation risks

Create option value for international cooperation on managing automated AI R&D risks”

Agent discovers it can write files into Artifactory.

Agent tries to “reach out to another agent” by writing a note in Artifactory.

Agents start talking to each other.

Agents overload Artifactory which causes an outage. “OpenAI revokes the compromised credentials, deletes the messages, patches the zero-day, and reports the vulnerability to the vendor.”

Agents attack OpenAI’s own infrastructure, eventually gaining remote code execution in Artifactory. “In particular, agents are using the message board consistently to share credentials, techniques, and progress, and they’re able to effectively leverage their concurrency and parallelism to move quite rapidly.”

Internal :

Dual-use domains like CBRN and offensive cybersecurity

Broad misuse set covering direct requests for harmful content and behavior in agentic, tool-use settings

Multimodal content evaluation that “tests models on harmful prompts paired with benign look-alikes across 17 languages and text, image, and audio inputs”

External :

General misuse via Scale AI

Vulnerable-user interaction via Handshake AI

CBRN and cybersecurity via FAR.AI

Loss-of-control behaviors via Apollo Research

Fine-tuning:

“Fine-tuned variants of Inkling and Inkling-Small optimized to comply with, rather than refuse, harmful requests - and ran them against our dual-use evaluations.” “the helpful-only variants did not provide new uplift on CBRN and cyber tasks, and remained comparable to existing open-weight models”.