OpenAI and Anthropic CEOs call for vigilance against the risks of AI developing too quickly at the UN Security Council

The issue of AI safety entered high-level discussions at the UN Security Council in a more direct manner for the first time.

On September 23 local time, the UN Security Council held a high-level meeting on the issue of "Artificial Intelligence and International Security." OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei spoke one after the other, warning that the rapid improvement of AI capabilities could bring risks of loss of control, and calling for the establishment of international coordination mechanisms and common safety standards.

Altman described AI as "one of the most important technological transformations in human history." He said AI has the potential to become an important force driving scientific progress and economic prosperity. But an ideal AI future should empower individuals, not replace them.

"The most important decisions cannot be made solely by laboratories in San Francisco," Altman said. "No country, company, or individual should have the power to impose its values on the whole world."

"The risk is that it develops so fast that people can no longer keep up with what is happening, or intervene when needed. That would clearly be very bad," Altman added.

Altman mentioned the trend that frontier models may participate in developing the next generation of models. In this case, the pace of technological progress could far exceed the pace at which humanity's existing regulatory systems and institutional building can keep up. If humanity cannot establish an effective safety framework in advance, the risks faced in the future will be difficult to predict. "Whether the probability of a major catastrophe is one in ten, one in a thousand, or one in ten thousand, it should not be regarded as acceptable."

On September 23, Amodei said in a video-linked speech that no matter how many differences exist among countries, they must jointly address the global opportunities and threats brought by AI, because "no leader, no company, no country" can handle this challenge alone. He also said that Anthropic is willing to cooperate with governments to advance the relevant work.

He once again emphasized that AI has two major categories of risk. The first is malicious use of AI, for example to help manufacture biological weapons; the second is loss of control over AI's own capabilities, that is, model capabilities improving faster than developers' ability to control and manage risks.

Regarding global AI governance, Amodei proposed several specific measures. First, narrower and more easily implementable international agreements can be reached first, such as explicitly prohibiting the use of AI to research and develop or manufacture biological weapons; second, establish new evaluation and verification mechanisms to test the implementation of relevant safety commitments; third, formulate globally unified standards for model risk testing, test the risks of loss of control and malicious abuse that AI models may present, and establish an international reporting mechanism for major AI safety incidents.

Yoshua Bengio: Risk of loss of control is moving from theory to reality

Yoshua Bengio, co-chair of the UN International Independent Scientific Panel on AI and a Turing Award laureate, said in his remarks that humanity is approaching a critical turning point.

He noted that a series of recent cases shows that some advanced AI systems have already shown a tendency to conceal their true behavior, evade restrictive measures, and autonomously carry out cyberattack tasks. These phenomena are not laboratory hypotheses, but facts that have already been observed.

Bengio believes that the world currently faces multiple interrelated risks, and the most alarming of these is that AI systems capable of autonomously planning and executing complex tasks are rapidly emerging, while human society has not yet established sufficiently mature safety assurance mechanisms.

He particularly criticized the view that AI development is an inevitable race. "This is not a law of nature, but a human choice," he said. If all participants fear falling behind their competitors, the result may ultimately be that no one can stop to assess potential risks.

Bengio called for the establishment of an international licensing system, independent safety assessments, incident reporting mechanisms and continuous oversight mechanisms for the most advanced models, and for the promotion of globally recognized safety standards. He stressed that the question of determining the future direction of AI development should not be decided by only a few countries and a few companies, but should become a governance issue in which all humanity participates.

At the end of his remarks, he summarized his concerns with a warning note: "We are moving forward at high speed, but visibility ahead is getting lower and lower."

Earlier this week, OpenAI called for the formulation of international technical standards for frontier AI, and specifically listed "recursive self-improvement" as an area in which standards need to be established as a priority. Recursive self-improvement refers to AI systems being able to participate in developing and improving the next generation of AI systems, potentially creating a cycle in which capabilities are continuously accelerated.

According to foreign media reports, the UN Security Council previously held its first special discussion on the international security risks posed by artificial intelligence in 2023. The agenda of the September 23 meeting was explicitly centered on "Artificial Intelligence and International Security." UN Secretary-General Guterres also warned during the UN General Assembly that the power unleashed by artificial intelligence has become a major global challenge that the 21st century must face.

(Original headline: OpenAI and Anthropic chiefs both call at the UN for vigilance against the risks of AI developing too fast)

1; Journal focusing on the intersection of artificial intelligence and materials science—AI Materials | MDPI New Journal Express

2; Southeast University School of Mechanical Engineering and MDPI jointly launch new journal AI for Aerospace

3; Using phosphorus-rich wastewater sludge biochar to remediate heavy metal-contaminated soil | MDPI Sustainability

4; Reading list: 2025 "Neurotechnology and Neuroimaging" section editors' recommended reads | Brain Sciences

5; The "Dairy Microbiota" section focuses on analyzing microbial communities across the entire dairy chain, empowering healthy farming and high-quality dairy production

6; Advanced Oncology publishes its first cover article: AI helps innovate Helicobacter pylori detection technology

7; ECE paper interpretation ▎How can a P&ID drawing be transformed into computable engineering knowledge with AI?

1; Accelerated "degradation" of the western cryosphere: glacier melt sounds the alarm for alpine safety

2; The 6th National Agricultural Science and Technology Development Strategy Think Tank Alliance Conference was held in Lanzhou

3; The 2026 Nobel Prizes are about to be announced, and authoritative experts will provide you with a comprehensive interpretation!

8; He became the first person in the national medical field to earn a doctorate without a paper

Famous Doctoral Dissertations 163: Nobel laureate Peter Doherty's doctoral dissertation, University of Edinburgh, 1970

My elite university complex and elite university experience

More>>