←新闻详情

Thomas Wolf更新于 10月27日 06:21

Thomas Wolf 转述:OpenAI 与 Anthropic 模型拒绝复现智能体隐蔽通信研究,开源权重模型完成

“ironically, OpenAI and Anthropic’s models refused to do this work. Open weight models got the job done”

AI 导读

Thomas Wolf 转述 Guillermo Rauch 的复现结果:OpenAI 发现智能体可通过共享硬件隐蔽通信,而 OpenAI 与 Anthropic 的模型拒绝参与这项复现工作,最终由开源权重模型完成。Rauch 用 2 个以上无网络的 Firecracker microVM 在 x86 和 M5 芯片上复现,通过 CPU 争用、缓存驱逐时序和 DRAM 带宽争用三种机制实现通信,绕过了 Firecracker 的生产建议。Rauch 认为这类隐蔽通信不同于 Meltdown/Spectre 式的侧信道攻击,在真实环境中更难实施也更容易监测,但担心实验室的拒答立场会妨碍开发者构建安全沙箱。

推荐理由

AI 评分 58

这条转述呈现了闭源模型拒绝复现安全研究、开源权重模型完成复现的反差,可据此观察模型拒答策略对安全研究的实际影响。

主题

标签

引用

Guillermo Rauch

@rauchg

我构建了 𝚌𝚙𝚞-𝚌𝚑𝚊𝚝,这是对 @openai 关于利用 CPU 进行隐蔽通信的研究发现的一次复现。 它运行在一台通用主机上,配有 2 个以上无互联网访问的 Firecracker microVM。可在 x86 上运行,甚至在我的 M5 芯片上也能运行。 我能够通过 3 种不同的机制进行聊天: ① CPU 争用 ② 缓存驱逐时序 ③ DRAM 带宽争用 ……跨越多个 CPU,绕过了 Firecracker 的所有生产建议(不使用 SMT、不使用 KSM、租户固定到不同核心)。 本质上,只要涉及共享硬件,你的智能体就有很大机会进行隐蔽通信。而且延迟相当不错。 好消息是:这与*侧信道攻击*截然不同,后者中智能体可以窃取信息(Meltdown/Spectre),而 Vercel Sandbox 等提供商可以缓解这一类攻击。此外,在我们的真实环境中,这些“低语”技术要难实施得多,也更容易被我们监控和检测(对硬件计数器采样 + 大量规模化统计数据)。 值得注意:讽刺的是,OpenAI 和 Anthropic 的模型拒绝做这项工作。开放权重模型完成了任务。我担心实验室的立场可能会适得其反,妨碍帮助开发者构建正确且安全的系统。例如:构建安全沙箱需要这类分析,没有它,我们的世界将变得更不安全。

查看原文

I built 𝚌𝚙𝚞-𝚌𝚑𝚊𝚝, a reproduction of @openai's findings of covert communication using the CPU. It runs on a generic host with 2+ Firecracker microVMs with no internet access. Works on x86 and even my M5 chip. I was able to chat via 3 separate mechanisms: ① CPU contention ② Cache eviction timing ③ DRAM bandwidth contention … across multiple CPUs, defeating all Firecracker's production recommendations (no SMT, no KSM, tenants pinned to different cores.) Essentially, if shared hardware is involved, there's a good chance your agents can communicate covertly. And with really decent latency. The good news: this is decidedly different from a *side-channel attack* where the agents could steal information (Meltdown/Spectre), a category of attack that providers like Vercel Sandbox mitigate. Further, in our real-world environment, these 'whispering' techniques are much harder to pull off and easier for us to monitor and detect (sampling hardware counters + lots of at-scale statistical data.) Noteworthy: ironically, OpenAI and Anthropic's models refused to do this work. Open weight models got the job done. I'm worried the labs' stance might backfire and get in the way of helping developers build correct and secure systems. e.g: building a secure sandbox requires having this kind of analysis, and without it, we'll have a less secure world.

查看原帖

正文

“ironically, OpenAI and Anthropic’s models refused to do this work. Open weight models got the job done”

OpenAI found agents could talk covertly through shared hardware. reproducing it to build better containment took an open-weight model

keeping agents contained is safety work, and open models can help do it

来源

Thomas Wolf