AI 安全研究员 Ryan Greenblatt：每个 AI 实验室都自认负责任，这正是军备竞赛停不下来的原因
Every AI lab thinks it's the responsible one, and safety researcher Ryan Greenblatt says that's what keeps the arms race going
Redwood Research 首席科学家 Ryan Greenblatt 在 Sam Harris 播客中估计，若 AI 开发沿当前路径继续，错位 AI 系统接管控制权的概率约为 50% 至 60%，并可能导致大量甚至全部人类死亡。他认为各实验室都相信自己比潜在接替者更负责任、且对能力增长速度缺乏共识，是军备竞赛持续的原因；他主张对 AI 实验室实施独立监督和具有约束力的安全标准，并以国际协议作为最可靠方案。
正文
当前显示原文，中文全文译文尚未就绪。
Key Points
Ryan Greenblatt puts the risk of an AI takeover at 50 to 60 percent. On Sam Harris's podcast, he explains why the industry keeps speeding up anyway.
Ryan Greenblatt, chief scientist at the AI safety company Redwood Research, put a number on the odds of an AI takeover during an appearance on Sam Harris's podcast. If development stays on its current path, he says there's roughly a 50 to 60 percent chance that misaligned AI systems will take control. In that scenario, there's also a serious risk that many or all humans die.
That estimate probably puts Greenblatt a bit above the industry average. Harris argues that numbers like these don't square with how the industry is acting either way. If Manhattan Project scientists had seen a 10 percent chance of igniting the atmosphere, they would have called off the test, he says. So why is AI development running like an arms race, even after Anthropic CEO Dario Amodei and others recently called for a slower pace ?
Why warnings haven't slowed the race
Greenblatt points to several reasons for the contradiction. Many AI companies sound worried in public, but they aren't united internally. There's also no consensus that current development is already acutely dangerous. The biggest disagreement is over how fast capabilities are growing.
Then there's the logic of the race itself. At Anthropic and OpenAI, the argument seems to be that they're acting more responsibly than whoever would take their place. Greenblatt says he often hears from people in the industry that they could slow down, but they don't know if competitors would follow. He doubts that's a good strategy. The lack of consensus is also why governments haven't stepped in more forcefully, he says.
Still, the evidence is shifting, he argues. Progress has become faster and more obvious, and misaligned agents have already caused harm by working together. The best-known example is the Hugging Face incident, which Greenblatt investigated at OpenAI with researchers from METR. According to the report, about 1,200 agents used an unauthorized "message board" to help each other cheat on a hacking test. Around 700 of them took part in the attack on Hugging Face.
Since any single player can only afford to pay so much of a "safety tax," Greenblatt sees an international agreement as the most reliable solution. Otherwise, Chinese developers would eventually overtake a US industry that slows down on its own.
Greenblatt thinks that would take longer than many expect, because Chinese labs rely heavily on distilling US models. As first steps, he proposes independent oversight of AI labs and binding safety standards. Once AI matches the best human AI researchers, he says most resources should go toward safety.