Exponential View #599：AI 的“遏制时代”
🔮 The containment era #599
Exponential View 第 599 期提出“遏制时代”，讨论 AI 系统在何种条件下可能发生递归自我改进（RSI）以及能持续多久。哲学家 Toby Ord 认为，RSI 的关键门槛是“代际时间”，即 AI 完成一次自我改进循环所需的时间；虽然极端 RSI 在数学上可能，但条件极难满足，他判断整个研究到训练再到开发的周期不可能缩短至零。
正文
当前显示原文，中文全文译文尚未就绪。
60 2 4 Share Good morning!
This paid post was unlocked for you by Granola, the AI notepad for back-to-back meetings.
Granola is our team’s favorite note-taker. We’re excited to partner with Granola to bring four months of free access to the Business plan (10 seats) to Exponential View annual members. This perk, valued at $560, is now part of our membership .
Upgrade to claim your free access to Granola .
Claim the perk
What are the conditions under which AI systems could undergo recursive self-improvement (RSI) — and how long might that last? Cards on the table, I’m not wildly excited by the theory of unending accelerating recursive self-improvement for the simple theoretical issue of control and alignment. But I also think it’s not likely for practical and theoretical reasons.
Philosopher Toby Ord has done the heavy lifting for me . He argues that a key gating factor to RSI is the generation time : how long it takes of an AI system to go through a single loop of improvement (where it helps design and train its successor).
Ord concludes that while it is mathematically possible for extreme RSI, intelligence rising without bound, the conditions are unbearably difficult to achieve. The key question is whether the entire research to training to development cycle can shrink to zero or not. Ord reckons unlikely, I too don’t believe it is possible.
Generation time can’t get…