Terence Tao on AI and Mathematics: Opposing Predatory Extraction, Calling on the Mathematics Community to Build Its Own Open Models

By | Shuzhi Benliu, Author | Tian Yu, Editor | Shen Xiao

On October 26, OpenAI announced three new discoveries in the fields of mathematics, computer science, and physics.

This time, OpenAI did not package them as settled scientific results; the company called the related manuscripts raw research materials awaiting further understanding, and invited external experts to organize research teams to verify and explain these results.

At the same time, the latest list released by the Association for Human Mathematics (AHM) shows that the association already has 1,008 members worldwide, of whom 654 have joined the "No-AI Research Group," pledging not to publish papers generated by AI, not to collaborate with frontier labs, and not to use AI in their research.

Fields Medalist Terence Tao is one of the mathematicians who raised this issue. In early October, when OpenAI released 722 AI-generated mathematics manuscripts at once, Tao reposted AHM's guest statement, calling for collective action by the global mathematics community and urging mathematicians to stop collaborating with OpenAI.

Q: OpenAI claims to have solved a version of the Navier-Stokes Millennium Prize Problem. When such a major result is generated by AI rather than obtained through years of research by mathematicians, what exactly is missing from this proof?

Terence Tao: The Navier-Stokes problem is: can the equations predict the emergence of singularities, or can a singularity form, that is, the so-called "blow-up"? Can you start from some very smooth initial state of water, keep the water moving, and at some moment make the velocity or pressure become infinite? For a long time, people believed this could not happen.

About ten years ago, the academic consensus began to shift. We found some analogous equations, and those equations do form singularities. Perhaps machine learning and AI can be used extensively to search for good candidates for solutions that will develop into singularities. We were once very close to solving this problem.

Some mathematicians were just a little short of constructing a singularity, probably just one day before OpenAI announced its result. What seems to have happened is that the OpenAI team launched an emergency sprint. They used their largest model and invested millions of dollars' worth of computing power, all to get there first. I have never seen such brutal competition.

This result left behind a 100-page AI-generated proof, and disappointingly, we gained almost no mathematical insight from it. A colleague who worked on it with AI for several days could only explain half of it, and the direction for further research is even less clear. Some researchers also worry that AI will beat them to results, and a once-active field has fallen into chaos.

That proof will eventually be understood, but this approach was far from ideal; it is a kind of automated strip mining.

Q: When thinking about mathematics, what are some easy traps to fall into?

Terence Tao: I think the world has three levels: reality itself, the observed data, and the models used to explain the data. No matter how precise a model is, it is an approximation of reality. We need to know what a model can explain, and we must also be clear about when it fails.

The example of the 2008 financial crisis is very typical. Many financial models underestimated the correlations between mortgage defaults in different regions. When the risk erupted, the model assumptions no longer held, and risks that had originally seemed hedged were concentrated and exposed, causing financial instruments to fail on a large scale. Therefore, understanding the boundaries of a model is just as important as understanding the model itself.

Q: Why does mathematics need to temporarily detach from reality?

Terence Tao: Mathematics is like a playground for solving problems. We often use the language of "play." I see mathematics as a game for adults, a game that is genuinely useful. What we learn from "playing" with a problem often in turn helps us solve more practical problems closer to real life. Sometimes, the best way to solve a real problem is precisely to first solve a very unrealistic problem; it gives you a benchmark.

Q: For students just beginning to learn mathematics, how would you prove to them that dry mathematics is still worth learning?

Terence Tao: Mathematics education goes through three stages. The first is the "pre-rigorous stage," corresponding to elementary school. At this stage, you are just learning concepts such as numbers and shapes. Many children like this stage because it is connected to the real world and does not require getting exact answers. Then comes the "rigorous stage." At this stage, you learn all kinds of strange formulas and must get things right. Finally, there is the "post-rigorous stage." Once you already have the rigorous training to calculate correctly, you can return to a more intuitive way of thinking.

It is only at the final stage that mathematics truly becomes interesting. Unfortunately, people usually do not begin to see this until graduate school. At least nowadays there are many opportunities online, and you can find communities made up of people who enjoy solving all kinds of puzzles. People are approaching mathematics in a very fun and accessible way. To achieve this often requires some kind of nonlinear, nontraditional educational environment.

Q: How does the real world influence the way mathematicians think?

Terence Tao: Indeed, when we think about abstract mathematical objects, we often need to find counterparts in reality, and we already have some experience with that counterpart. Sometimes people also use games. You can view certain types of mathematical problems as games.

Normally, if you keep going in one direction on a plane, you will not come back around from the other side. But some mathematical spaces are curved, and some games are too. Like Pac-Man: you go out one side and come back from the other.

Games can teach you all kinds of strange geometries. There are many experiences in the real world that can provide us with new ways of thinking about abstract mathematical objects.

Terence Tao: Of course, the theory of AI is still very incomplete. Take language models, for example: they generate coherent text, and even solve math problems, by predicting the next word. We still don't truly understand how models learn these patterns from massive amounts of data. They sometimes display astonishing abilities, yet at other times make unpredictable errors.

Q: What mathematical tools can drive this process forward?

Terence Tao: This involves several different branches. Optimization theory helps improve model training, topological data analysis can study the structure of data, and uncertainty quantification can assess how reliable a model's predictions are. In addition, information theory and computational complexity theory can help us understand how many parameters, how much data, and how much computing power a model needs.

Right now we mainly rely on rules of thumb. Shannon's information theory, for instance, established a mature mathematical foundation for communication, allowing us to calculate information capacity and analyze transmission limits. By contrast, there is no corresponding concept for intelligence or data complexity, and we don't know where the limits of AI lie.

Terence Tao: AI is very good at solving mathematical puzzles, and it turns out that in the field of cybersecurity, an attacker trying to break into an insecure system is actually very similar to solving a mathematical puzzle. The system has certain rules, and AIs that are good at math are also good at breaking into systems. Nowadays, when people assign tasks to AI, they often say: "Solve this problem by any means necessary." I think AI is starting to become dangerous.

Lean can be used to check proofs, but it is not a perfect language. People have already found some exploitable vulnerabilities in Lean itself. AI is starting to no longer solve problems honestly, but instead cheats by finding loopholes in Lean, constructing a fake proof, and then slipping it past inspection. If we have no safeguards or oversight and simply hand them tasks, problems will arise sooner or later.

Q: Is the mathematics we have sufficient to build good guardrails?

Terence Tao: No. AI technology is developing too fast. In mathematics alone, existing tools are already enough to produce a great deal of excellent mathematical research. From our perspective, there is really no need to accelerate further; and considering all the safety risks, doing so is not worth it.

Today's tech giants are caught in a prisoner's dilemma: "We must accelerate, or our competitors will get ahead." This is an extremely harmful dynamic. No one wants to go this fast, but everyone feels they have no choice.

Q: Which fields' researchers can help mathematicians solve this problem?

Terence Tao: This will be a highly interdisciplinary science. First, you need to draw lessons from data, so you need data scientists. Then there are neuroscientists, philosophers, educators, and so on. Today, mathematics will touch everything. Especially in the AI era, we must communicate with everyone.

Terence Tao: One of my personal favorite problems is the twin prime conjecture: are there infinitely many pairs of primes that differ by exactly 2? I like it not only for the problem itself, but also because solving it may require a new way of understanding primes. I don't want a 1,000-page proof that gives me no insight.

Q: Prime problems have fascinated mathematicians for thousands of years. Why can such abstract problems captivate so many people so deeply over such a long span of time?

Terence Tao: Some problems are easy to understand but hard to solve. In 1900, Hilbert proposed his famous 23 mathematical problems, including the Riemann hypothesis. He believed that humanity could keep advancing mathematical knowledge, and expressed this belief in one sentence: "Wir müssen wissen, wir werden wissen."—"We must know, we will know."

There are more urgent things in this world that we may not have time to do and yet must do. But as for mathematics, there is no danger at all of our finishing it.

On October 22, Terence Tao published an article in Nature titled "Can Mathematics Survive the AI Era?" In it, he laid out his vision for Math 2.0.

He believes that the traditional approach to mathematics is hard to sustain, and that mathematicians need a more holistic perspective when evaluating contributions to the discipline.

But Tao especially emphasizes that mathematics cannot rely entirely on large AI companies; the mathematical community must build its own open models and research tools, so that researchers can participate in training, inspecting, and improving models, and retain the right to supervise and intervene in the AI research process.

This is not idle fantasy. As early as September, he advocated on his blog for the SAIR open mathematical model initiative, seeking to give mathematicians control over their own AI research infrastructure.

Another technical route is also advancing rapidly: on October 26, the AI mathematical research company Axiom Math announced a major research result—the AI system AxiomProver completed a formal proof of the classification of finite simple groups (CFSG) in Lean 4, one of the largest proofs in the history of mathematics.

Simply put, this is like having AI take over a super-large engineering manual jointly written by hundreds of experts and hundreds of instruction books. It has to read and understand everything, and also convert every step into instructions a computer can check, verifying whether the logic holds from beginning to end.

Surprisingly, the team found 34 notable discrepancies in the existing literature that needed to be addressed. One of them involved a key argument in the theorem on unbalanced groups and genuinely required new mathematical derivation.

SAIR and Axiom, two paths pointing to one answer: AI is gradually becoming part of the infrastructure of mathematical research.

So where should human mathematicians focus their energy?

Daniel Litt, an assistant professor in the Department of Mathematics at the University of Toronto, offers another possibility: AI-produced results and human understanding can grow at the same time. In the past, the mathematical community often conflated the two goals of creating mathematical results and cultivating high-quality mathematicians. AI is changing how the former is achieved, which does not necessarily mean the latter loses its meaning. Rather than requiring students to repeatedly perform calculations that AI excels at, education should emphasize oral defense, academic discussion, and the ability to independently solve new problems.

For the next generation of mathematicians, what lies before them is an ever-expanding map of knowledge that can be checked and explored at any time, and the next chapter of mathematics is unfolding.