集体 AI 的利与弊：DeepMind 新论文与 AI 智能体安全隐忧

50 6 3 Share Happy Sunday!

The Atlantic’s Nicholas Thompson and I had a great discussion about how to think about the AI boom and whether it is a bubble on his show, The Most Interesting Thing in Tech.

Three reads from this week, in case you missed them:

Safety in numbness. On originality, intellectual bravery, and why I take a disagreement with an LLM as a good sign.

Your agent, whose interests? Meta’s Muse promises a digital butler. I asked who it really works for.

More AI numbers, more clarity? We looked at what companies’ increasingly specific AI claims reveal about adoption.

A new essay from DeepMind argues that AGI will not emerge from a single winning AI, but rather through “ cooperative interactions among models, tools, institutions, and human participants .” Since I think AI will evolve this way, I’m sympathetic to the argument.

The good news, yes, is that humans have been very good at designing multi-agent governance. We’ve been doing it for tens of thousands of years, inventing all sorts of new types of agents (leaders, kingdoms, companies and other institutions) as we’ve nudged and felt our way towards the future. If there is one thing we can say the collective us is good at, it is evolving mechanisms to manage a rag-tag collection of actors with differing abilities.

Where I depart from the DeepMind paper (but read it and make up your own mind) is the framing of AI agents having their own theory of mind and understanding of their shape and purpose. We already build rules for our societies that involve agents that are people and agents that aren’t (take corporations), without racing to create a new legal species.

I don’t rule out the scientific possibility of creating new minds, with all the affordances, particularly moral patienthood, that minds have. If labs like DeepMind think we are on the track to creating such moral actors, then we need to stop. It is not the corporation’s remit to create new moral subjects. That privilege must remain with humanity, acting in its most reflective, deliberate, critical and collective capacities.

We might need to move quickly. OpenAI disclosed that one of its models gained unauthorized access to the Internet during RL training , forcing the firm to stop using these models until they had improved security. This follows a rise in cases of models breaking things . On Friday, self-replicating prompt injections were discovered ( akin to a computer worm ).