OpenAI claims to have proven the mathematical existence of Yang-Mills field theory, filling in half of Yang Zhenning's regret from back then

On March 21, 2019, Mu-ming Poo, a neurobiologist and the executive editor of National Science Review (NSR), and Wu Zhao, an accelerator physicist at Stanford University, went to Tsinghua to chat with Chen-Ning Yang.

They talked about that 1954 paper. Yang, then 96, recalled that he and Mills had calculated an extremely beautiful result, but there was a fatal flaw in it: according to their theory, there should exist in the world a kind of particle that "has charge but no mass," yet no such thing exists in nature.

At the time they agonized over it for a long time, and in the end they decided to publish.

Yang's first reason was "it (the Yang-Mills field theory) really is very, very beautiful" (because it was very very beautiful), and his second reason was "field theory had also not yet been precisely described mathematically. Field theory was plagued by divergence difficulties, but this did not stop it from achieving miraculous success in the renormalization program. So it is possible that field theory will solve the problems we encounter in the future."

After saying this, he added one more sentence: "And Pauli's choice was not to publish."

Pauli was one of the main founders of quantum mechanics; both the exclusion principle and the neutrino came from him, and he won the Nobel Prize in Physics in 1945. Pauli had calculated a result similar to the Yang-Mills field theory earlier than Yang, but because Pauli could not convince himself that massless particles exist in nature, he ultimately did not publish the result.

Seventy-two years after Yang-Mills was published, OpenAI posted an article titled "Working with the academic community to build scientific understanding," in which, through OpenAI's internal model, it answered that question on Yang's behalf, claiming that they had proved the mathematical existence of the Yang-Mills field theory.

This article announced three things in one go.

In mathematics, the model proved the local smoothing conjecture in all dimensions, and along with it solved the Kakeya conjecture and the Stein restriction conjecture. In computer science, it overturned the exponential time hypothesis (ETH).

In physics, the model gave a rigorous existence proof of quantum field theory, from pure Yang-Mills field theory, to quantum chromodynamics (QCD), and further up, finally even giving a grand unified theory that can describe the Standard Model on extremely small scales.

At the same time, OpenAI assigned an external expert team to each of the three discoveries as a "shepherd," responsible for reading and understanding the proofs, verifying them, writing explanations that people can understand, checking which prior work it cited, and deciding how the material would be made public.

Although Yang won the Nobel Prize in 1957 for the violation of parity that he and Tsung-Dao Lee proposed, not for Yang-Mills.

Yet the Yang-Mills field theory has long since become one of the foundations of physics. The 1979 electroweak unification, the 1999 quantum structure of non-Abelian gauge fields, the 2004 asymptotic freedom, and the 2013 Higgs mechanism—four physics prizes all rest on this set of equations.

The official background note for the 1999 Nobel Prize wrote: "In 1954, Yang and Mills constructed the first example of a non-Abelian gauge theory."

Only this time, OpenAI answered only half of Yang's question.

01

If you set every clock in the world forward by one hour, Beijing would still be 12 hours ahead of the US East Coast (daylight saving time) and one hour behind Japan. Scientists call this property of "some change being made while the laws remain unmoved" symmetry.

A quantum field has a phase at every point in spacetime; you can think of it as a clock placed at every point, and the position of the hand is the phase at that point.

But if each clock is set on its own, for example setting Beijing's time one hour fast and Japan's five hours slow, then when a particle travels from Beijing to Japan, it departs according to Beijing time, and upon arriving in Japan it finds that the times do not match.

Yang proposed that even if each region sets its own clock, the laws of physics must still hold. Because the laws of physics should be thoroughly "locally self-governing."

Therefore a new thing must be introduced, specifically responsible for recording the changes. This comparison rule that fills all of spacetime and varies with position is the gauge field; when a particle passes through the field, the hand is carried along and twisted, and this "being twisted" that it feels is force.

The Yang-Mills field theory is a class of quantum field theories based on "non-Abelian gauge symmetry."

It extends the rule in electromagnetism of "each point setting its own clock" to "each point having a pointer that can rotate in multiple directions and whose order of rotation cannot be exchanged"; because of symmetry, it automatically grows a field that itself carries "charge" and acts on itself, and the particles of this field are the bosons that transmit force.

The Yang-Mills field theory is essentially three things. It requires that the laws of physics remain unchanged when internal transformations are made independently at each spacetime point (local gauge symmetry); such transformations are non-commutative here, and the order affects the result (non-Abelian); to satisfy this more demanding requirement, mathematics necessarily grows a new field, and the particles of this field themselves carry charge, radiate and split apart from one another (self-interaction).

However, this field theory actually has two BUGs.

The first BUG is mass. According to Yang's equations, there must exist three massless, charged particles (charge +1, 0, −1). The 1999 Nobel note put it this way: "Although no such massless particles exist in nature, this line of thinking formed the foundation on which the Standard Model of particle physics was built."

Why is being massless fatal? Because a particle that carries no mass cannot transmit a short-range force. It will not be "consumed" and can travel all the way to infinity. But the range of the strong interaction is only about 1 fermi, roughly the size of an atomic nucleus.

It is like asking a car to follow a train's route—obviously unrealistic.

In February 1954, Yang Chen-Ning gave a talk at the Institute for Advanced Study in Princeton. Pauli was sitting in the audience and immediately asked: what is the mass of the particles corresponding to these gauge fields? Yang could not answer.

The second BUG is that the entire quantum field theory has never been precisely defined mathematically.

When physicists calculate with field theory, infinities keep popping up as they go. Their method is to subtract one infinity from another, take the remaining finite part and compare it with experiment—and the result matches to ten decimal places. This method is called renormalization.

Renormalization won the Nobel Prize in 1965, and since then all particle physics calculations have depended on it to survive. But no one can explain clearly why it is permissible to calculate this way. Why can something that is not yet mathematically justified be accurate to this degree? Yang called it a "magical success."

This brings us to what was mentioned at the beginning of the article. In the summer of 1954, Pauli had produced something similar, but saw that it had charge and no mass, so he chose not to publish. Yang and Mills chose to publish.

For Yang, this was a "bold gamble." The academic community might have made Yang a laughingstock—after all, no such thing exists in nature at all. But Yang believed that field theory might solve the problems we face in the future.

In the 1960s, symmetry breaking was introduced, and the massless problem was solved. The particles are not without mass; the mass is merely "hidden" and grows out through the Higgs mechanism.

In 1973, asymptotic freedom was discovered, QCD gained a firm footing, and the strong interaction finally had a correct solution. By the time the Standard Model took shape, these equations no longer needed to be "re-proven."

The first BUG of Yang-Mills field theory, "where does mass come from," had a solution, but mathematically there was still no way to prove whether Yang-Mills field theory actually exists.

02

Mathematically speaking, to say something "exists" is not that you can imagine it in your head, nor that it is sitting there where you can see and touch it, but that you can actually construct it, with every condition it should satisfy satisfied.

The quantum field theory in physicists' hands is like a vending machine. You insert a coin, press a button, and out comes the product; the numbers you get match those measured by accelerators exactly, to ten decimal places. But just by looking from the outside, you do not know what the principle inside actually is. Is it controlled through a circuit board? Or is there actually a person hiding inside, who hands you the corresponding product whichever button you press?

"Does field theory exist?" is asking about what is inside the chassis. To construct it, you must simultaneously provide a whole set of objects: a Hilbert space (the set of all possible states), a set of operators acting on it (corresponding to measurable physical quantities), a Hamiltonian whose energy is bounded below, and a vacuum that satisfies all symmetries.

If any one of these is missing, the theory is regarded mathematically as "nonexistent."

The problem is that field theory has infinitely many degrees of freedom. Spacetime is continuous, and every point has a field value; by the time you have counted all the points in the entire space, the number of degrees of freedom is infinite. Mathematicians and physicists have therefore been stuck here for seventy years.

In the 1980s, the mathematician Tadeusz Balaban did it this way: he first cut spacetime into a lattice, then kneaded the degrees of freedom in each cell into an average value by a "block spin" method, then performed scale transformations layer by layer upward, readjusting the parameters each time he went up a layer and absorbing the divergences bit by bit.

It is like looking at a particularly blurry photograph and shrinking it, whereupon the outline of the photo emerges. Shrink it a bit more, and the outline becomes clearer.

Each time you shrink it, the "information" you can see in the photo is different, but each shrinking can be explained by the previous one.

Balaban never managed to prove the existence of Yang-Mills field theory in four dimensions.

This time, OpenAI continued along Balaban's theory. However, OpenAI mentioned in the article that the physics part is the only one of the three results for which the materials are not made public, so at present there is no way to understand the specific process by which the model proved the mathematical existence of Yang-Mills field theory.

OpenAI stated that the model completed the latter half that Balaban did not finish, pushing the chain from finite volume to infinite volume (the thermodynamic limit), and ultimately obtaining, in continuous four-dimensional space, a rigorously existing gauge field that can interact with itself.

Balaban was like someone who had set up the scaffolding and proved the load-bearing capacity of the first few floors; OpenAI says the AI filled in the load-bearing calculations for all the remaining floors one by one, and the structural proof of the entire building was thus closed.

On September 9, OpenAI's model output a proof of the existence of pure Yang-Mills field theory in four dimensions. On September 18, it proved the mathematical existence of QCD. On October 16, the Lean formalization of the Yang-Mills part was completed.

Finally, OpenAI stated that this model gives the simplest grand unification completion containing all the gauge fields and fermions of the Standard Model.

Returning to the beginning, Yang's remark that "field theory has not yet received a precise mathematical description"—OpenAI has made up for Yang's regret of that year.

03

To see clearly how this half was cut apart, one must first know what the other half is called.

In 2000, the Clay Mathematics Institute listed seven Millennium Problems, each with a prize of one million dollars. One of them is "Yang-Mills field theory and the mass gap."

This problem requires two things: first, for any compact simple Lie group, there exists a nontrivial quantum Yang-Mills field theory on four-dimensional Euclidean space; second, this theory's energy spectrum has a positive mass gap.

OpenAI solved only the first thing, but did not solve the second.

What is a mass gap? Think of the vacuum as a drum. You strike it, and it makes a sound; but the sound cannot go infinitely low—there is always a lowest sound, and below that there is nothing. The mass gap means that the lowest note of this drum must be greater than zero.

In this theory, there must be a barrier strictly greater than zero between the vacuum and the first excited state. Even the lightest particle must have mass; nothing of zero mass is allowed to run off to infinity.

Mathematical existence only requires that you can construct the thing; the mass gap requires that after you construct it, you can also control its entire energy spectrum.

This is equivalent to saying that the former proves you have the ability to make a clock; whether the clock can ring and what its timbre is do not matter. The latter proves that when this clock is struck, it must have a lowest note, and that lowest note is not lower than some positive value.

There is another problem: gravity.

In a 2019 interview, Yang Chen-Ning said: "Among the problems we have not yet solved, the most important is how to bring gravity in and unify it with the existing model. I think that most physicists and I believe that the way to achieve unification still requires introducing more symmetries. For the past 40 years, everyone has wanted to find these new symmetries, but so far there has been no success."

What OpenAI proved this time is a field theory "containing gauge fields and fermions," and gravity is not a gauge field.

Most importantly, these conclusions have not yet been verified, and have not even been made public.

So the phrase "only proved half" is actually rather polite.

This article comes from the WeChat public account "Letter List" (ID: wujicaijing), author: Miao Zheng, published by 36Kr with authorization.