一条推文如何改变 AI 安全辩论

Most people think AI is risky, but they don’t agree what to do next.

It was the tweet heard ‘round the world.

Last week, a little-known Anthropic employee named Jacob Coxon announced that he was resigning from the company. “I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly,” he wrote. “The people building AI earnestly believe that it could kill us all by the end of the decade.”

Coxon wasn’t the first employee to quit over similar fears. In 2024, William Saunders told a US Senate committee that he “resigned from OpenAI because I lost faith that by themselves they will make responsible decisions about AGI.” And in February, Mrinank Sharma wrote that he was leaving Anthropic because “the world is in peril” from AI and other risks, and he felt he couldn’t act with sufficient integrity at the company.

Yet for whatever reason, it was Coxon’s message that broke through. His tweet has been viewed over 170 million times. He was interviewed on CNN, CBS, and Fox News. Lawmakers and celebrities alike referenced his resignation. AI risk was suddenly a mainstream topic.

It’s not entirely surprising that more people are paying attention now. Since the start of summer, there’s been a steady drumbeat of news about AI companies struggling to keep their agents under control. And Americans are already critical of other aspects of AI development — such as the mass construction of data centers.

Still, it was a bit shocking to suddenly see discussions of AI risk well outside the tech bubble. Musician Sheryl Crow mentioned Coxon’s departure in an Instagram post. TMZ asked Matt Damon about human extinction because of AI. Joe Rogan interviewed prominent AI safety advocate Daniel Kokotajlo.

For some AI safety advocates, this was the crucial moment they’ve been waiting for. In a Monday LessWrong post titled “Crunch Time is Now. Drop Everything,” the head of PauseAI, Maxime Fournes, wrote that “this is the kind of chance we have dreamed of, and we have a responsibility not to waste it.”

But it’s unclear how increased popular interest in AI safety will be channeled. Fournes wants a global treaty pausing development of frontier AI systems. That seems unlikely, at least in the short term. Donald Trump is scheduled to meet with Chinese officials next week, and AI safety is on the agenda. But recent comments from both sides suggest that they are not close to reaching a deal.

In short, no one is going to force AI labs to change how they do business in the next few months. Despite that, some company leaders have signaled a willingness to slow down voluntarily — especially if their rivals do the same.

Keep reading with a 7-day free trial