Import AI 462：超级说服、自我维持的 AI 与通往 ASI 的路径

Algorithmic paradigm shift: In the same way that Transformer and Mixture-of-Experts architectures jumped the field forward many years, the same thing could occur again with other fundamental innovations. We could imagine, for instance, advances in adaptive computation at test-time or deployment, or overcoming the limitations of today’s context windows. If we made advances here or in other areas this could be a big deal, but it’s inherently hard to reason about - akin to trying to anticipate things that could expand our understanding of the nature of reality prior to the invention of general relativity.

Recursive self-improvement : It could be possible for AI systems to build their own successor systems. If this is the case, then we could rapidly transition from general intelligences to superintelligences. There are some wildcards here - personally, it’s obvious to me that today’s AI systems are speeding up human researchers in creating future AIs, so a kind of “co-creation RSI” loop has started, but AI systems don’t (yet) exhibit the kind of paradigm-changing creativity which seems required to move the frontier forward in significant steps. It’s unclear how much this happens - even without this kind of high-bar creativity we might be able to have systems grind out marginally better versions of themselves and get a slow compounding process going. Capabilities could explode or they could taper out or “anything in-between”.

Things that inspired this story: Thoughts about how a grand negotiation between machines and people might one day take place; how every truly important negotiation has two personalities involved in it; the Sentience Accords. Thanks for reading.