AI safety researcher Suarez says there is a greater-than-even chance AI causes human extinction within ten years, and that he has not contributed to retirement savings for ten years
English text is being prepared.
AI alignment researcher Suarez judges that there is a greater-than-even probability that AI will cause human extinction within the next ten years, and says he has not added to his retirement savings for ten years. He believes the risk comes from the possibility that superintelligence may not care about human survival, and that once it can autonomously obtain energy and chips and build automated infrastructure that does not depend on humans, humans will find it difficult to constrain it. He advocates that governments stop training artificial general intelligence systems and shift to specific uses such as cancer research and safer autonomous driving.
Body
苏亚雷斯长期研究人工智能对齐问题，也就是让强大AI系统的行为符合人类意图。他曾在Google和微软担任工程师，并与他人合著《如果有人造出它，所有人都会死》。他对风险的担忧并非认为AI会憎恨人类，而是认为超级智能系统可能根本不在意人类是否生存；如果人类妨碍其目标，或试图关闭它、开发竞争系统，人类就可能被视为障碍。
在他设想的风险路径中，关键并不只是AI能否发动攻击，而是它能否变得自给自足：如果系统能够自主获取能源、芯片和其他资源，并建立不再依赖人类的自动化基础设施，人类对它的制约能力就可能下降。苏亚雷斯形容，人们未必能预先知道超级智能会以哪种具体方式造成灾难，但仍可能判断自己正面对极强的对手。
他主张政府停止训练通用人工智能系统，把研发重点转向癌症研究、更安全的自动驾驶等具体用途，并表示灾难并非不可避免，自己仍在尽力阻止这种结果。他强调的解决办法，是不要造出可能杀死所有人的系统。
苏亚雷斯的判断是个人风险评估，并非已证实的科学预测，也不代表人工智能研究者之间存在统一共识。他停止增加退休储蓄的说法，反映的是他对未来风险的看法，不应被理解为对他人理财决策的建议。