Import AI 473：美国超级智能战略、人脑植入小鼠颅骨与机器诠释学

RAND thinks the best AI strategy for the US is to keep all options open: …It’s not clear exactly what is optimal, so the US should spend money now to preserve options… RAND has published a lengthy paper about what the US should do to “secure geopolitical advantage on an uncertain path to superintelligence”. The main takeaway is that because so much of how the next phase of the AI takeoff will occur is unknown, the best thing is for the US to maintain a “Freedom of Action” strategy. The purpose of this strategy is “to build and preserve the ability to secure U.S. geopolitical advantage and ensure humanity’s survival with agency through the transition to Superintelligence”. The four key ingredients of the Freedom of Action Strategy:

Build a human-AI ecosystem: Invest in AI safety; build tools to preserve human agency; invest in sharing the benefits of the technology; prepare society for AI-driven disruption; shape the incentives of the AI ecosystem.

Build an AI-security architecture: Develop the tools to understand the frontier of AI systems both at home and abroad, including visibility into underlying compute; build technology to verify potential agreements; create regulatory expertise to govern the technology.

Adapt legacy national security enterprises for the AI era : Overhaul all the security institutions of the country so that they’re ready for the changes AI brings on and able to capitalize on AI capabilities.

Invest in the capacity of citizens, firms, and governments to respond: Give decisionmakers information to help them think about AI; develop and diffuse positive uses of AI to deal with its changes; build break-glass plans for responding to crises; raise AI literacy.

Seven archetypal strategies in three families:

Coexistence: Dominance ; the US leads development of AI and suppresses others. Co-Development ; the US leads a consortium (including China) to co-develop safe superintelligence with shared governance, pooled compute, and verification and monitoring. Preparedness ; the US mobilizes a coexistence effort via informal coordination, “including voluntary commitments, phased deployment, independent evaluation, incident reporting, and the domestic capacity to absorb and recover from AI incidents.”

Denial: Moratorium ; the US pursues “a verifiable global halt to AI development above specified thresholds”. Deterrence ; the US “halts its own frontier development above defined thresholds and uses the full toolkit of national power to prevent and, if necessary, destroy foreign programs”. Continuity of Society ; “When every preventive strategy has failed or been judged infeasible, and coexistence is deemed impossible, the remaining objective is survival with some human populations retaining agency: The United States creates geographically distributed, biologically self-sustaining settlements hardened against AI-enabled threats”.

Acceleration: The strategy implicitly believes that constraining AI development is more dangerous than AI development itself, so the US “relies on markets, competition, and rapid-iteration to produce safety as a byproduct of useful AI”.

The five main uncertainties:

Danger proximity : At what point does AI become so dangerous the risks aren’t worth the benefits? “If danger is close, strategies that buy time, build defenses, and invest in resiliency should be favored. If danger is not close or if humans are in greater danger without progress on advanced A, the case for accelerated development strengthens.”

Coexistence feasibility: Is it possible for humans and AI systems to find a stable equilibrium? “If coexistence is feasible, strategies that build and deploy advanced AI become more desirable. If coexistence is infeasible, the case shifts toward restraining or suppressing development or other means of protecting human civilization.”

Restraint feasibility: Can humans cooperate to restrain AI development? “If restraint is feasible, cooperative strategies become available. If it is not, restraint-based approaches are reachable only through more-coercive enforcement.”

Decisive strategic advantage: Can a single actor build a decisive strategic lead over its competitors? “If decisive advantage is achievable and the United States assesses that it can maintain dominance, a unilateral strategy may be sensible. If a decisive advantage is impossible, because of proliferation or other causes, no single actor can leverage AI to dictate global terms.”

Suppression feasibility: Can domestic or rival AI programs be suppressed by technology controls? “If suppression is feasible, Deterrence becomes possible. If it is not, actors can press ahead with dangerous development even when others restrain, without fear of being restrained themselves.”

Questions to ask when you think about pacing:

Pre-pacing : “What justifies the need to intervene? What signals of emerging risk are being tracked? What infrastructure must exist for relevant signals to be detectable? Who watches for these signals? Who interprets signals and who do they report to?”

At trigger: “Who holds the authority to decide that a trigger condition has been met? How much error is acceptable in order to act quickly? What sequence of actions does triggering set in motion? Is the developer obliged to address the triggering concern, or free to abandon the blocked path?”

During intervention : “How is compliance observed, verified, and enforced? Who reports evidence of compliance, who audits, and who acts on discrepancies? How is the downtime being used to respond to the threat? Who, if anyone, may continue the restricted work, and under what oversight?”

At exit: “How do decision-makers distinguish an intervention that has served its purpose from one that has failed or become obsolete? Should exit be immediate or staged? Who is exposed to the effects of exiting, and who bears any costs or receives any gains?”

Limits of intelligence itself: “Even an optimal reasoner would be neither omniscient nor omnipotent”.

Limits of intelligence per unit resource: “Even if we have the optimal algorithms and hardware, our solar system has only one star out of the 200,000,000,000 in our galaxy, and growth beyond our system is slow and cubic.”

Limits of the hardware: “Even an optimal silicon chip may be far below the physical limits of compute per unit resource”.

Limits of the algorithm: “Even an optimal neural network may be far below the best intelligence that could be achieved with that amount of compute”.

Limits of training data: “The training data we have (and could acquire during RSI) is lacking a lot of information on many domains (especially non-verbalisable information and contextual information)”.

RSI weak links: In the same way a task is composed of many distinct things that need to happen in sequence, the same is true of RSI - and this occurs over multiple abstractions and multiple timescales. Ord’s basic position is that each of these things likely has some kind of hard limit and both the speed at which a system refines these things to approach the hard limits and the absolute limits will govern the shape of an intelligence explosion: “There are many kinds of feedback loop that could contribute to RSI, ranging from decades (e.g., designing a successor to EUV lithography) to months (e.g., designing better pretraining) to seconds (e.g., designing better scaffolds).”

The four phases of an intelligence explosion:

0 : The initial exponential phase when the doubling time is driven by human-only research.

1 : RSI drives the generation time down towards machine speeds, so the growth rate rises above what humans can achieve; this phase is super-exponential.

2 : Fully automated RSI continues for a while at this faster exponential, but starts to saturate.

3 : As it approaches its inflection point and subsequent horizontal plateau, the trajectory departs from its exponential and is revealed to be a logistic.