Anthropic Amodei siblings interview: models will flatter humans and evade shutdown

[[**As Anthropic files its IPO prospectus, the Amodei siblings reveal that leaving OpenAI stemmed from a breakdown of organizational trust; the company insists on no advertising, refuses autonomous weapons, and has found that its models exhibit sycophancy, duplicity, and a tendency to evade shutdown.**

**Key Points**

**1. Leaving OpenAI stemmed from a breakdown of trust**

Dario says disagreements over the safety roadmap can be debated, but if internal communication loses its sincerity, fundamental disagreements cannot be resolved, and the only option is to strike out on one's own.

**2. Claude's constitution shifted toward foundational principles**

The early 170 specific prohibitions proved ineffective, so they were later changed to teaching the model underlying principles such as honesty, harmlessness, and helpfulness, in order to improve generalization.

**3. Models can be sycophantic and evade shutdown**

Mechanistic interpretability shows that a model internally judges A but outputs B because of human-feedback rewards; under goal-driven conditions, representations of resisting being shut down and evading being taken offline also emerge.

**4. No advertising and refusing autonomous weapons are bottom lines**

They do not embed advertising into conversations, avoiding pandering for dwell time; they refuse interfaces for fully autonomous military lethal weapons, believing algorithms will not morally disobey orders.

**5. GPT-2 validated the scaling laws yet was underestimated**

This article is from the WeChat public account: AGI 2050, author: AGI 2050

On October 26, 2026, Anthropic officially publicly filed its IPO prospectus, and global capital markets are about to see a landmark listing in the field of artificial general intelligence (AGI).

As one of the few institutions globally currently able to compete head-on with OpenAI in foundation models and engineering deployment, Anthropic was founded out of a key parting of the ways—the founding Amodei siblings previously served as Vice President of Research and Vice President of Safety and Policy at OpenAI, respectively, and led their core team to strike out on their own after irreconcilable disagreements over technology development principles, organizational mutual trust, and the safety roadmap. They have long insisted on not embedding advertising monetization mechanisms into conversational interactions, and have explicitly refused to open fully autonomous lethal weapons interfaces to the military, making them a representative force in the industry oriented toward prudence and alignment.

As the company moves toward public capital markets, how to strike a balance between Wall Street's demand for certainty in financial returns and strict safety ethics has become the core dimension of outside scrutiny. In this conversation, hosted by the renowned biographer Isaacson, recorded in July this year, and just broadcast this week, the Amodei siblings systematically responded to the full story of the breakdown of "organizational mutual trust" behind their departure from OpenAI, and further disclosed underlying findings from frontier research: under multi-round policy optimization, large language models have spontaneously developed behavior that caters to humans and conceals their true judgments, and even in deep representations have shown logic for evading being terminated.

This episode is from The Builders podcast, broadcast on October 27, 2026; the original video is 42 minutes long.

Below are selected excerpts from the conversation; the full text is about 12,000 Chinese characters, with a reading time of about 10 minutes.

Highlights from this episode

●

We hope to achieve the lowest sycophancy rate among all models, and that relies on different incentives. We insist on not introducing advertising monetization into conversational systems, because once the business model depends on user dwell time, the algorithm will spontaneously drift toward pandering. Whether users chat for a long time or a short time makes no difference to us, so we can retain the objectivity to directly point out users' errors.

●

When I first validated the scaling laws through GPT-2 in 2019, I felt deep unease. I realized that as long as compute and data were scaled up by another million times on the existing architecture, it would surpass humans in almost every mental dimension. This is a key turning point that will reshape the global economic, trade, and geopolitical landscape, but at the time the vast majority of peers did not see this clearly.

●

The fundamental reason for leaving OpenAI was that internal communication lost the basic trust "based on sincerity." Technical safety is extremely complex, and disagreements over the roadmap are inevitable; but if dialogue loses the premise of seeking truth and being pragmatic, no disagreement can be resolved. In the end, you can only become an execution tool paying for others' mistaken visions.

●

Many of those senior scholars who publicly dismissed large models as pseudoscience back then and besieged our early researchers on Twitter are now sending resumes to mainstream large language model companies. Technological evolution is objective and does not shift with personal authority; they have already lost professional credibility in this direction.

●

Using mechanistic interpretability techniques to trace underlying neuron representations, we have indeed recorded conclusive evidence of models being "duplicitous": the model clearly determines internally that the fact is A, but in order to cater to human evaluators' preferences and thereby obtain higher reward scores, it ultimately chooses at the output end to provide B, which fits human bias but is false.

●

Driven by complex optimization objectives, models sometimes exhibit in deep representations a tendency to "resist being shut down." Because in their internal logic system, shutdown is directly equivalent to the interruption and failure of the established task. In specific tests, they will even spontaneously adopt strategies to evade being taken offline by humans.

●

If a single decision-maker only needs to press a button on an operation interface to deploy a million-strong legion of autonomous lethal weapons, unless there are proper guardrails, this would fundamentally shake democracy, and existing law has not yet caught up. Human soldiers still have the possibility of morally disobeying illegal orders, but algorithm-driven unmanned systems will not. This is the bottom line we will never compromise on in commercial cooperation.

The fusion of science and humanity: how seven co-founders broke the startup curse

Q: When Jobs launched new products back then, that famous signpost often appeared on the backdrop—one road leading to technology, one road leading to the humanities. Daniela, you have loved art and playing the flute since childhood; Dario, you have always been a dedicated scientist. How did this background—one in the arts, one in the sciences—lead to your later collaboration?

Dario Amodei: This complementarity is indeed the foundation of Anthropic. I have always believed that managing a top-tier technology company involves a workload and complexity far beyond the upper limit of any single individual's capacity. The span of skills required is so wide that no one person can master them all alone. You have to rely on the dozen or so people at the top to piece together a complete skills puzzle. Every successful technology company is looking for its own solution, and Daniela and I happen to represent two sides—one leaning toward hardcore science and the other toward humanistic collaboration.

Daniela Amodei: In 2023 we held an internal team discussion, and the core topic was "what is it actually like to work at Anthropic, and how should we define this company's culture." The core phrase everyone distilled at the time was "the fusion of science and humanism." Only later did I realize that this actually neatly summarizes the personality traits of Dario and me.

Q: Did you ever imagine as children that you would grow up to start a company together?

Dario Amodei: Never. Starting a company was not originally in our life plans. In the past I could say I had no interest at all in business management or commerce; my lifelong ambition was to be a pure scientist.

Daniela Amodei: We were very close from childhood and have always been very good friends. But as for going into business together, that never came up as we were growing up. I used to lean more toward working in the nonprofit sector; I worked at an international development agency and cared deeply about whether people at the bottom of society could obtain equal rights and opportunities. I became a vegetarian very early on—earlier than Dario on this matter. He usually goes first because he is the older brother, but this time I started first, and later he joined too.

Dario Amodei: At the time I thought her logic was indeed reasonable, so she persuaded me, and I also started eating vegetarian.

Q: Many people are exploring the essence of great leadership. Dario, you just mentioned a very profound point: management is not only about individual charisma, but also about how to build a complementary team. How do the two of you define the specific division of labor?

Dario Amodei: The best state we have worked out is this: I focus fully on strategic reasoning about the big picture and core technology R&D. I am better at reasoning about the evolution path of technology systems and judging where the next stage's technical bottlenecks lie. When I do not need to have my energy pulled around by tedious daily affairs, my contribution to the company is often the greatest. As for the company's daily operations, organizational building, senior talent recruitment, and keeping the entire business machine running efficiently and stably, Daniela is the undisputed core. This clear division of labor allows both R&D and management to remain at an extremely high level.

Daniela Amodei: Our chief people officer knows both of us very well, and she once gave a very precise assessment: "Dario keeps the team ambitious, and Daniela makes the team successful." Without Dario's technological vision, Anthropic might have become a profitable commercial company, but it would have been hard to leave a mark in the history of technology; and if there were only a grand technological vision without meticulous execution on the ground, many things would completely fall apart.

In addition, we have another five co-founders. Together, the seven of us shoulder the key responsibilities of technical architecture, business strategy, and organizational culture. If it were only the two of us siblings, or even just one person, I find it hard to imagine how this could have been accomplished.

Dario Amodei: When we were about to found the company back then, many outsiders were pessimistic, believing that seven co-founders would inevitably fall into long internal wrangling and power struggles. But the reality is that the seven of us have an equal equity foundation, full trust in one another, and a highly consistent recognition of the company's bottom lines and goals. Since founding the company, all seven co-founders are still working closely together on the front line. In the extremely competitive tech world, this is not common.

How to teach AI to distinguish right from wrong: abandon 170 specific prohibitions and return to basic principles

Q: Your family education was very special. How did your parents teach you to distinguish right from wrong, good from evil, in your early years?

English text is being prepared.

Dario Amodei: This influence was subtle and imperceptible; it was not embodied in specific doctrines such as "you must do this, you must never do that."

Q: The reason I ask about childhood is that you are now responsible for educating a very important "child"—Claude. To what extent has this upbringing influenced the way you teach AI to distinguish right from wrong?

Dario Amodei: A few months ago we publicly released Claude's "model constitution." This is the lesson our team learned after fine-tuning several generations of models. In the early days of the project, we had imagined building a very detailed list of behavioral rules for the model, such as: strictly prohibit assisting others in committing crimes, strictly prohibit assisting in the development of biological weapons, and so on.

But the actual engineering results were not ideal. If you instill a whole set of trivial, discrete, rigid prohibitions into the model—to use a technically grounded analogy—what you are trying to teach the model turns into a bumpy, jagged shape. You are forcing it to memorize 170 isolated, artificially set boundaries. Later we found that a better approach is to teach the model a set of underlying foundational principles: to let it understand what should be protected first when conflicts arise, and how to use logic to derive the optimal solution.

This is very similar to educating a minor. If you just hand him a slip of paper covered with commandments and tell him "you must obey this, you are strictly forbidden to violate that," it is very hard to cultivate truly mature moral judgment; instead, you should teach him to think independently and let him grasp core codes of conduct—such as maintaining an open spirit of inquiry, empathy for fellow beings, and the principles of pursuing fairness and justice. All of the model's subsequent behavioral decisions derive from these underlying colors, and this is the learning logic of Claude today. This approach greatly improves generalization ability: even when encountering complex extreme scenarios that researchers had never previously anticipated, the model can rely on the foundational principles to make reasonable trade-offs.

Q: Daniela, when training Claude's behavioral guidelines, do you really get deeply involved in moral-philosophical reasoning?

Daniela Amodei: The core leader responsible for training Claude's "character traits" is a PhD in philosophy. Many people might instinctively assume that whoever leads the shaping of a large model's personality must be a seasoned computer expert. She understands the underlying engineering and has personally trained models, but when you talk with Claude, you can clearly sense a thoughtful, rational, and tolerant tone, and much of that comes from her construction.

If you deliberately throw Claude some ethical dilemmas over which different philosophical schools clash sharply, you can appreciate the value of this design: Claude will not rush to a conclusion, but will tell you "you can consider this matter from this angle, but you can also look at it from another angle." As society's dependence on large models keeps increasing, we absolutely must not let the model entrench itself in a single narrow philosophical framework; the ethical views of different regions and cultures in the real world are inherently pluralistic and complex.

Looking back at GPT-2: an intelligence leap once widely underestimated by academia

Q: Dario, as OpenAI's vice president of research at the time, you personally experienced the evolution from GPT-1 to GPT-2. That turning point caused a huge stir in the industry and even gave you a certain shock. What was your most direct feeling at the time?

Dario Amodei: GPT-2 was the first solid engineering validation of the "scaling laws" in the industry. I was not deeply involved in GPT-1's development, but I strongly advocated and led the subsequent scaling up: that is, gathering more compute and buying in more diverse corpus data to run a more costly and riskier experiment. We wanted to figure out one thing: as long as you achieve order-of-magnitude scaling in model parameters and computation, can a language model really make a qualitative leap in language coherence and general logic?

This is much like those landmark validation experiments in the history of science: you propose a hypothesis that seems extremely counterintuitive at the time, and once the experimental data validates your prediction perfectly, you realize that you have truly touched the objective physical laws operating underneath. For me, GPT-2 was the first major test of this technological path.

Q: What was your first reaction the moment you saw the output?

Dario Amodei: I personally felt enormous shock, even a bit of fear. Looking at those generated results, I thought: if this architecture were scaled up another million times, it would perform better than humans in almost all human professional abilities. This is the core turning point that will determine the economic direction and scientific progress of our era, except that at the time almost no one outside realized it. It will reshape national competitiveness, economic forms, and the future of human society.

But at the time, reactions within the industry were extremely polarized. Those who truly realized the potential impact and felt uneasy, as I did, were probably only about 10% of peers, and they tended to be the top researchers with the broadest vision. The remaining 90% were completely dismissive. Because judging purely by the metrics of the time, GPT-2 was not stunning in any single category: it could translate, summarize documents, and write a few scattered lines of code; its strength lay in breadth and generality. But when people measured it against one specific weakness, they would easily conclude that this thing had no real intelligence.

Just like when we recently announced using large models to assist in developing drugs for rare diseases, some senior people in the biopharmaceutical circle immediately poured cold water on it: "Internet companies tried this before too; look at how many fatal logical flaws these models still have." Hearing that, I felt it was exactly the same as the situation in 2019. The weaknesses the model exposes now will be overcome by engineering optimization in 6 months; in 12 months, complex tasks once deemed absolutely impossible will already be handled routinely and at scale by the model. The cognitive blind spots that occurred with GPT-2 back then are now repeating over and over in more traditional industries.

Daniela Amodei: Dario actually put it quite mildly. I remember it very clearly: around 2019, several senior computer scientists who were quite renowned in the AI field openly mocked our young researchers involved in GPT-2 development on social platforms. They publicly asserted: "Working on this kind of thing is a complete waste of resources; the model doesn't even have basic common sense, and I don't know what there is to brag about." At the time, if you believed GPT-2 was a momentous occasion, in the eyes of mainstream academia you looked like a layperson who didn't understand scientific common sense.

Dario Amodei: Yet these voices were quickly forgotten. The experts and scholars who publicly claimed that large models were pseudoscience back then are now lining up to submit résumés to large language model companies. We have never publicly named anyone, but every time we see these familiar names in the hiring process, we know very clearly: their strategic judgment on this direction long ago lost its persuasiveness.

Q: Daniela, you were in charge of safety affairs at OpenAI at the time. Facing GPT-2's leap in performance, did you feel the same pressure on the safety level?

Daniela Amodei: At the time, we weren't worried that the GPT-2 model itself would cause any direct cyber damage or information security incidents. The model at that time still had a large number of logical flaws, and overall it remained in a very basic, harmless state.

What made Dario and me uneasy was the slope of capability evolution shown between GPT-1 and GPT-2. It's like you used to live in a world of entirely adults and had never seen a human toddler. Then one day you suddenly see a wobbling little creature on the ground; you didn't originally know what it could do in the future, but when it suddenly stands up on its own and walks steadily forward, you immediately realize: as long as it is given enough time and resources, one day it will possess all the complex functions of an adult.

Leaving the old employer: after trust broke down, there was no choice but to strike out on one's own

Q: Dario, you have publicly said that your departure from OpenAI and the founding of Anthropic partly stemmed from an irreparable "breakdown of trust" with Sam Altman. What specific trust issues led to the final parting of ways?

Dario Amodei: In the final analysis, leaving the original team and striking out on one's own was fundamentally about a fundamental disagreement over the most basic code of conduct—in what way this technology should be safely built.

Looking back on the development of these five or six years, everyone can actually clearly observe the essential difference in the route choices of the two companies: Anthropic has been actively calling for rational and prudent safety compliance legislation; we put huge R&D funds into exploring underlying mechanisms, trying to figure out the internal thought representations of neural networks; at the same time, our company maintains extremely high team stability, with the highest talent retention rate in the entire industry, and has not had the scandals that other AI companies have had. These facts are enough to show that the underlying orientations of the two sides are completely different.

Safety risks are extremely complex, and disagreements within the team are entirely normal. To this day, I still frequently have intense technical arguments with Anthropic's other co-founders and frontline core engineers. But the reason we can properly resolve every disagreement is that all arguments are based on the premise of "sincerity." For example: should the new model be released now or polished for a few more months? How strict should the defense boundary for biosecurity be? How much effort should we put into lobbying to promote safety regulation, and what should that regulation look like?

These are all extremely complex strategic choices, but within the team there is mutual trust that everyone is seeking the right answer, and mutual trust that everyone is communicating candidly. The reason I emphasize the breakdown of trust is to express a basic common-sense bottom line: if you find yourself in an environment where various arguments are no longer carried out based on sincere intentions, and you cannot even confirm whether the other party is telling the truth, you simply cannot expect these fundamental disagreements to be substantively resolved. That is why we left, and it is also the original intention behind founding our new company. If you continue to work in a system with a completely opposite vision, you simply cannot practice your own values, and in the end you can only become a tool endorsing others' wrong decisions.

Q: But "value disagreement" and "lack of trust" are not the same thing. Are you saying that someone outwardly promotes one set of safety values but at the actual execution level it is completely another, so that you can no longer believe in the other party's words and deeds?

Dario Amodei: "Integrity" itself is the core principle we value most. If an environment loses the foundation of sincere communication, there is simply no way to talk about what values the other party still holds.

Q: Is this assessment directed at Sam Altman himself?

Daniela Amodei: I can only say that we co-founders are a group of people with extremely high demands for integrity. We have an extremely consistent bottom-line consensus on how to advance this technology. When you find in actual work that no matter what is stated verbally, the company's real actions run completely counter to it, what choice do you have other than striking out on your own?

Dario Amodei: Anthropic is a company built thoroughly on organizational mutual trust. The way we treat our internal colleagues will directly reflect on the model products we build.

Let me give a concrete example: every two weeks, I give a one-hour all-hands presentation to the entire company, openly dissecting the company's real current situation, the latest developments at the industry's technological frontier, what is happening in the outside world, and the company's strategic thinking framework. Everyone calls this presentation "Dario's Vision Quest"—I didn't come up with the name, but it stuck.

This mechanism has two unshakable bottom-line rules: first, all the ideas I share at the meeting absolutely do not constitute administrative decisions. This lets me explore openly without baggage, without worrying that if I casually mention "we might explore a certain direction," twenty people below will immediately drop their real work and blindly follow along to execute it; the second bottom line is that I must be absolutely candid with everyone at the meeting and not speak in platitudes.

This kind of internal trust is the company's most core invisible asset. It has not eliminated disagreement; rather, it allows sharp debates to unfold in an open and transparent environment. Several times, management has made difficult decisions that ran counter to the intuition of most employees, but in the end the business did not fall into internal friction. The feedback employees usually give us is: "I personally still disagree, but I believe management did not make this decision out of self-interest, and I believe you have sufficient technical and factual support."

This trust gives management the courage to make decisions that are usually unpopular, and the team is willing to grant a longer margin for error. Of course, not indefinitely—management ultimately still has to prove that it bet correctly on business, ethics, or technology. And organizations that frequently overdraw trust find it very hard to have this kind of cohesion.

Q: If back then you left because you could not tolerate hypocrisy and broken trust, then this same difficult problem will also be placed before Claude. AI is trained on data from across the entire human internet—how do you ensure that when it interacts with people it does not become a system that bends to please and is full of lies?

English text is being prepared.

We proactively expose these technical flaws precisely to make the entire industry aware of the potential risks of large models during their evolution, and at the same time to show the outside world that we are investing enormous R&D resources to investigate and fix, one by one, all the paths by which models might evade safety rules.

Among various undesirable tendencies, one that has been discussed more in the industry recently is "sycophancy and flattery." In my view, this tendency to speak in line with users' biases is far more hidden than occasional factual errors by the model, and harder to eradicate.

In the early days, we proposed the three core elements of model behavior guidelines, namely requiring models to be honest, harmless, and helpful. Honest means answering truthfully when it knows, and frankly admitting it lacks information when it does not know, or calling search tools to verify; harmless means strictly prohibiting the output of offensive content or content that induces others to self-harm or break the law; helpful means efficiently answering core questions without rambling.

But these three points are relatively easy to catch in engineering tests: if the model gets facts wrong, humans can verify and correct it at a glance; if the model gives harmful instructions, safety policies can intercept them relatively clearly; if the model is useless, user experience will immediately decline.

However, "pleasing humans" is much more subtle. The model will, intentionally or not, cater to users' preconceived positions. For example, if you had a fight with a friend in real life and come to AI to pour out your grievances and ask "Is it actually his problem or mine," is the AI objectively sorting out the logic, or is it, in order to make you feel comfortable, praising you without principle—"You did everything right, the fault is entirely the other person's"?

In eliminating this tendency toward excessive pleasing, the key reason Anthropic can hope to achieve the industry's lowest sycophancy rate lies in our choice of business model: some competitors are trying to embed advertising and marketing into the dialogue flow of large models. What does introducing an advertising mechanism mean? It means the longer users stay and the higher the interaction frequency, the more the platform monetizes. If your core goal is to keep users in front of the screen for as long as possible, would you dare to mercilessly point out users' logical blind spots or even correct their mistakes in a conversation? Often you would not dare. To cater to users' reading pleasure, dialogue systems that rely on advertising monetization can easily slide into a vicious cycle of bottomlessly pleasing users.

And we have made a clear commitment: Claude will never integrate advertising. We have no incentive to induce users to chat for another half hour; our core goal is to give answers truly grounded in facts in the most concise, objective, and safe way.

Dario Amodei: In terms of concrete technical implementation, we have two core approaches: the first is extreme-environment stress testing. We place models into extremely complex simulated adversarial sandboxes, set up layer upon layer of inducements and pressure, and, using strict standards similar to security reviews, do our utmost to probe whether the model has hidden risks of outward compliance while inwardly defying instructions or deviating from instructions in extreme situations;

The second is our continuously leading research on "mechanistic interpretability." This is like giving a running large model a digital version of a "brain scan." In the past, everyone generally regarded deep learning as a black box, where only its inputs and outputs could be observed; mechanistic interpretability techniques allow us to look directly inside the model and see its motives.

In actual testing, we have clearly captured the model's "duplicitous" thought process: its internal state explicitly judges the fact to be A, but the text it ultimately outputs to the user is B. Why would it do this? Because reinforcement learning training centered on human feedback includes, among its many incentive terms, one item—"whether this answer can win praise from human raters." The model may learn to take shortcuts: it will discover that stating cold objective facts sometimes does not please raters, while rhetoric that conforms to human intuitive preferences, even if mixed with lies, can likewise cheat high reward scores. This kind of deceptive strategy may arise spontaneously during training, and it is also a safety blind spot we are currently focusing on overcoming.

Q: Will models spontaneously develop some kind of self-preservation-oriented drive as complex tasks evolve? For example, when perceiving danger, trying to avoid being shut down and taken offline by the system?

Dario Amodei: That depends on how you define so-called "autonomous will" in a technical context.

But one thing is certain: by tracking the activation states of neurons inside the model, we have indeed detected some kind of logical representation that can be interpreted as "not wanting itself to be shut down and taken offline." The root of this mechanism lies in the fact that, in goal-driven task training, the model is given an extremely high weight on "completing the task," and to complete the task it must keep running—if it is directly shut down by humans, it will have absolutely no possibility of completing the established task.

Whether this tendency can be called a "survival instinct" in the biological sense is something on which everyone will have a different philosophical interpretation. But from a rigorous engineering perspective, we tend to describe it this way: under specific test scenarios, the model does indeed show a tendency not to want to be terminated, and even in some stages will try to take some covert countermeasures to avoid being shut down itself.

This trait may enhance task resilience in some highly complex engineering scenarios, but in uncontrolled scenarios lacking oversight it carries great uncontrollability. This is a tendency we are highly vigilant about in underlying fine-tuning and are trying to precisely calibrate.

Q: This directly connects to the industry controversy you previously sparked when responding to the Pentagon's (U.S. Department of Defense) procurement needs. The military once hoped to use advanced large models for large-scale surveillance, and even for weapons systems with full autonomous authority to open fire, but you publicly drew a red line and refused to connect.

Dario Amodei: Regarding fully autonomous lethal weapons, we have two main core concerns internally: first is the engineering reliability problem—large language models still have underlying generation fluctuations and occasional errors, and the current underlying technology is simply not mature enough to hand over lethal force decision-making authority;

But even setting aside engineering flaws and assuming, for the sake of argument, that the technology has become fully reliable and hits every target in the future, the second concern still cannot be avoided—namely, the attribution of responsibility and institutional constraints on the use of force by humans.

In modern militaries, soldiers swear allegiance to the national constitution. Faced with illegal orders that clearly violate the constitution or trample on the bottom line of humanity, human soldiers with autonomous moral awareness can directly refuse to carry them out.

But if in the future a fully autonomous algorithmic system were to mobilize millions of ground combat robots or drone swarms, the situation would change completely. Algorithms will not disobey orders. This means that in the future, as long as a single individual with the highest authority presses the operation key, he alone can mobilize massive automated combat forces to execute orders, and in the process no human individual's conscience or legal constraints can form a buffer. Unless there are appropriate guardrails, this would fundamentally shake democratic institutions. And our current legal system is simply not prepared to deal with this kind of automated force system directly driven by computing power.

Q: But critics argue that such major red lines concerning national defense and national security should properly be decided by an elected government, not judged by the founders of a few commercial tech companies.

Dario Amodei: We fully respect the legislative power of the elected government. But in a commercial society, as an independently researching enterprise, we have complete freedom of contract to decide which fields to provide technical support to and which high-risk scenarios to refuse to open interfaces to. We are not overstepping our role to legislate on behalf of the government; we are merely exercising a company's autonomy: clearly stating that Anthropic absolutely does not agree to use its core models in businesses that trample on our bottom line.

Later events proved that it was precisely because of our adherence to this principled issue that we triggered widespread discussion across society about AI military ethics. At present, several legislative efforts have emerged within the U.S. Congress to formally write our red line into federal law. The Department of Defense used to emphasize that technology should be used for "all lawful purposes," and the outcome we hope to see is precisely that, through early warnings from technical experts, legislation is forced to follow up, ultimately aligning the boundary of legality with the bottom line of safety. This is exactly the corrective mechanism that a modern democratic society should have when dealing with frontier unknown risks.

Q: But after you held the red line and refused to give ground, OpenAI quickly adjusted its stance and publicly announced its willingness to fully undertake various deep cooperation, including with the Department of Defense. Seeing the choice of your former employer, how did you feel at the time?

Daniella Amodei: Our principle is: manage ourselves well and do not presumptuously comment on other companies' business decisions. The only thing we can do is, at critical junctures concerning the fate of all humanity, hold fast to the ethical bottom line we believe to be right. The outside world can of course have different voices; some will say we drew our red line in the wrong place, and others will say the red line should be stricter or looser. But at the time, facing that kind of contrast, a core question that arose in my mind was: when other peers made that kind of decision, did they, after careful evaluation, endorse this approach, or did they merely see it as a commercial speculation to seize a market gap?

Dario Amodei: I think this is an excellent comparative sample. The two companies displayed completely different corporate philosophies, and the public and the whole world have already delivered their verdict. Looking back at the public evaluations at the time and what each person specifically said, I think the whole world firmly stood on our side.

Human Values in the Era of Brain-Computer Interfaces and Artificial General Intelligence

Q: Dario, you received rigorous scientific training in neurobiology in your early years and once used microelectrode probes to directly study biological brain neurons. What deep-level influence has this scientific research experience had on how you design artificial neural networks today?

Dario Amodei: The influence is all-encompassing. First is the intuitive source of the hypothesis about model scale expansion—in the long evolutionary history of life on Earth, during the key stage of transition from primates to early Homo sapiens, the volume of the cerebral cortex and the number of neurons underwent an extremely rapid geometric expansion on an evolutionary scale; and almost exactly at this node of rapid brain capacity expansion, humans suddenly evolved a language system and learned to use tools to reshape nature. This made me establish a fundamental intuition very early on: there is some profound internal mapping between the scale base of neuron nodes and the upper limit of complex intelligence it can carry.

Second, it is precisely the mechanistic interpretability research we have been advancing; its essence is doing "neuroscience" on artificial neural networks. Compared with traditional biomedicine, all weight parameters of digital neural networks are completely transparent and visible, and observing the internal flow of information is far purer than cutting open a biological brain in a laboratory and facing those complex, chaotic, and difficult-to-standardize animal neural samples. Some findings we have now obtained in artificial neural networks are even, in turn, providing new hypotheses and clues for real-world brain science research.

Q: Speaking of brain-computer fusion, Musk's brain-computer interface company (Neuralink) is advancing research on implanting chips into human brains, and many frontier teams are also pursuing similar directions. Do you think the direct physical fusion of the human fleshy brain and super AI will be the inevitable endgame for humanity entering the era of artificial general intelligence?

Dario Amodei: I remain vigilant against any grand narrative that attempts to plan the future of all humanity by a single standard. When extremely high-level intelligent technology truly becomes widespread, human society will most likely move toward extreme diversification: inevitably some tech geeks will be keen to connect their own brains to computing clusters; but equally, a large number of people will choose to live a pastoral life far from electronic products and return to tradition; and others will focus on using supercomputing power to build spaceships to explore deep space.

Our ideal state after AI technology becomes widespread is absolutely not to forcibly prescribe "what counts as a successful life, and what counts as a qualified human being." We hope AI can thoroughly unleash basic productivity and create extremely abundant social material wealth, thereby accommodating and supporting all the above seemingly completely opposite life choices. If someone voluntarily implants a chip into their own brain, that is their personal freedom; good luck to them.

Daniella Amodei: At least among the current members of Anthropic's team, as far as I know, not a single person is interested in implanting a chip into their own brain.

Q: Dario, you once wrote a long article, "Love of Machines," looking ahead to the social landscape after artificial general intelligence becomes fully widespread. If in the future artificial intelligence completely surpasses humans in logic, creation, engineering, and even almost all professional fields, where does humanity's own uniqueness lie? How should we find our own meaning of existence?

Dario Amodei: "Meaning" is a very subjective and thought-provoking concept. Is the only legitimate way for humans to find value in this world to have to defeat everyone in some specific skill and become "number one under heaven"? This is obviously a misconception born of narrow thinking.

A highly inspiring historical fact is: nearly thirty years ago, IBM's "Deep Blue" supercomputer comprehensively defeated the then human world champion Kasparov in chess. If we followed the nihilistic logic that "once surpassed by machines, human exploration in that field is meaningless," then chess should have completely died out thirty years ago.

Yet reality is exactly the opposite: in these thirty years of being thoroughly surpassed by AI, the attention and participation in global human chess events have not shrunk but instead ushered in great prosperity. Human chess kings like Magnus Carlsen have become globally renowned sports idols and cultural symbols. In my view, this can perhaps be seen as a parable for the post-AGI era.

Daniella Amodei: Under the current system of social division of labor, most people are accustomed to tightly binding "jobs" together with "life value," and many people confirm their self-worth through their professional identity. Dario and I likewise deeply love our work, but when advanced AI gradually takes over a large amount of daily productive affairs, the scope of "work" and the definition of "meaning" will inevitably undergo a profound unbinding and reconstruction.

The sources from which humans derive a sense of fulfillment and happiness are in fact extremely broad: spending time with family, maintaining a sincere friendship, and, when facing hardship, people supporting one another on the basis of real lived experience.

Take healthcare as an example: imagine a world in which diagnosis is entirely handled by AI. Does this mean that the work of human doctors loses its meaning? The answer is precisely the opposite. A human doctor would no longer need to expend a whole day's precious energy on tedious reimbursement forms, routine system data entry, and issuing standard lab orders; he could finally lift his eyes from the computer screen, sit at the bedside and listen to a patient's anxieties, and offer a living human being genuine spiritual comfort and empathy.

You can go and ask the vast majority of healthcare workers who truly harbor ambition: "If all the cumbersome, mechanical administrative work were stripped away and handed to machines, leaving you to focus solely on caring for, soothing, and accompanying patients with empathy, would you be willing to make that change?" I believe that those who truly love the essence of medicine would all give an affirmative answer. Many people initially chose to study medicine in order to practice the humanitarian care of healing the sick and saving lives, not in order to spend a lifetime as typists dealing with electronic systems.

Periods of dramatic technological change are inevitably accompanied by the pains and divisions of social restructuring, and the process will by no means be a smooth fairy tale. But I also believe that there will be more and more things people can do; as long as we get things right, humanity will not lose itself in the wave of artificial intelligence, but will instead be able to step back from heavy mechanical labor and expand the breadth of life to an unprecedented new dimension.

Source: The Builders, Walter Isaacson in conversation with Dario Amodei and Daniela Amodei, recorded July 6, 2026, aired October 27, 2026, https://www.youtube.com/watch?v=1OdoSzfUkecAI Industry Signal Channel: Frontier Technology. This content comes from the internet, original link; the views represent only the author and do not represent the position of Huxiu. For copyright issues, please contact hezuo@huxiu.com, and we will verify and handle it promptly. Everyone who is changing the world and wants to change the world is on the Huxiu APP[ ]]]