Altman: PC memory utilization is only 5%, DRAM should be prioritized for AI data centers

The conversation had reached that point.

This Thursday, in an interview with the YouTube channel Cheeky Pint, OpenAI CEO Sam Altman drank beer throughout and chatted in a relaxed atmosphere about local inference of large AI models, and the resulting problem of insufficient memory in home computers, and inadvertently dropped a bombshell.

As we know, the memory price surge triggered by the current global AI wave has already caused PC and phone prices to skyrocket, and individual consumers have long found it hard to bear. At the same time, capacity expansion by the major memory manufacturers will take a while. Just as frontier AI companies are taking the lion's share of memory, they have already begun discussing whether memory in personal computers is being fully utilized.

Altman mentioned a hypothetical scenario in which people could run local AI on their own computers, which might be machines they already own, or ones they want to assemble because of the rise of large AI model technology. He considered this a bad idea.

"From a global resource allocation perspective, having everyone leave the world's limited DRAM idle in their own computers while utilization is only 5% is actually very bad," Altman said. "If you want to work in an environment where this resource supply is extremely tight, then in the cloud you should maximize resource utilization as much as possible. But somehow, many people's intuition is exactly the opposite."

Today, OpenAI's main business is based on closed-source models running in large-scale data centers, and his defense of his company's business model sparked a debate in the AI community.

It also raised concerns for many: in the future we may find it even harder to own our own hardware or software, and may only be able to consider renting cloud-based agents and computers.

Although ordinary PCs, phones and AI accelerator cards use different types of memory, their production competes for some upstream wafer capacity. Altman was discussing the allocation of global DRAM resources, and when it comes to who is qualified to decide where memory capacity is used, most of the time it is naturally the customer placing the larger order.

In fact, this interview was long, and Altman also talked about the PHP vulnerability that sparked global panic over the past two weeks, sandboxing of trained models, and AI research in the field of mathematics.

Let's see what he mainly said:

Regarding the super-huge PHP vulnerability, Altman's judgment was: the ability of large models to discover vulnerabilities continues to improve rapidly, and the code surface on the internet is extremely vast. When people begin systematically using AI to scan all code, "we will find ourselves in a world where severe vulnerabilities far exceed expectations." He believes patching alone is not enough and that the architecture must be refactored; routers, operating systems and the application layer all need to be redesigned for a world where "vulnerabilities will become increasingly intelligent." Two years ago, "rewriting the entire tech stack" still sounded crazy; now startups aiming to build an entirely new stack have already emerged.

He rarely offered industry self-criticism: "I don't think we have done a good job explaining to people why all the potential downside risks of this technology are worth it." He said the AI industry's narrative is bad salesmanship, promising to cure cancer and reduce costs on one hand while depicting a future of unemployment, loss of control and the need for strong regulation on the other. He reiterated that OpenAI insists on iterative deployment and confronting capability overhang, on the grounds that "a few square miles of people in San Francisco" must not be allowed to make all the decisions.

When asked whether a "maximum-security prison" could be built for models, Altman gave a pessimistic answer: sandboxes are only effective to a certain extent, and monitoring (including chain-of-thought review and activation monitoring) can go one step further, but ultimately there must be a high degree of trust in alignment. He asked in return: when facing a model a million times smarter than the smartest security researcher, one capable of discovering new science that humans do not yet understand, would you really dare to trust that sandbox?

He made clear that he hates the term "agent swarms," because swarm evokes locusts, and insisted on calling them agent teams. He believes that the property of mutual checks among teams corresponds precisely to the mechanism of achieving robustness through pluralism, and is a safety direction worth pursuing.

The cost of an intelligence explosion and scientific discovery

Altman sketched a curve: in the GPT-4 era, it was hard to effectively invest even $1 of compute into a single problem; chain of thought lets models "think hard," and now we can run an entire agent team in parallel tens of thousands of times. The usage gap between ordinary users and the most advanced users is constantly widening.

On how OpenAI allocates compute internally, he said employees enjoy great freedom, but high token consumption is never rewarded: "A high ranking on the token leaderboard is actually a bit embarrassing, and definitely not a reason for promotion." He used the proof of the Navier-Stokes equations as an example: a single proof cost about $10 million, and during that period things "went out of control for a week," because at the time no one knew anything about the scaling laws of agent teams. But he expects that a year later, with smarter models and methods, it could be redone at 1/100 of the cost.

He also uncharacteristically showed emotion: the situation at the time was truly extraordinary—on the weekend when a new model was launched internally, everyone threw the toughest problems at it, and it managed to solve them one by one. People had not expected the model to be able to do these things at all. Together, everyone witnessed some of the hardest research problems in many people's academic careers being solved by this model one after another, across multiple different fields—the scene was simply astonishing.

“For two nights I barely slept at all, because I was so excited that I simply couldn't sleep. Many of the results we later published were in fact the problems that were solved at that time.”

Altman felt incredibly excited and happy at the time, and at the same time an unexpected sadness, melancholy, and a sense of “an era is coming to an end” welled up. It was not just nostalgia, but also a kind of empathy for human scientists' “hearing the Way in the morning, dying content in the evening.” He insists that humanity deserves access to basic knowledge, but the industry owes the practitioners in these fields a solution.

Although it is still too early to talk about superintelligence, Altman believes that a trillion slightly superhuman intelligences are better than one with a trillion IQ:

"I would much rather intuitively have a trillion AIs of slightly superhuman intelligence join society than one AI of one trillion IQ join society."

His argument is: society's strong robustness comes from a large number of members checking and balancing and cooperating with one another; a “single-brain” world does not have this kind of resilience. But he stresses that this will not happen automatically and must be actively pushed forward.

On the issue of the global memory shortage, Altman described OpenAI's layout: in-house chips (improving intelligence performance per watt), self-built data centers, robotics projects, and an even bolder concept, “self-replicating data centers,” using AI to control robots to operate fabs, build turbines and solar panels, and construct data centers. He said, “This is already within our grasp.”

When asked why “the market regulating itself” has been so slow to materialize, he said current progress is terrible and laid out the behavior logic of memory manufacturers: this industry has long been a hard business of investing $20 billion in capex to earn $1.9 billion, and historically giants have repeatedly come close to bankruptcy because of overbuilding factories. Today each company can earn about $100 billion in free cash flow a year, but the painful memories make them extremely conservative about expanding production. He therefore asserted that “the invisible hand is now failing.”

On the real level of recursive self-improvement (RSI), Altman said models can now complete tasks such as “finding a 5% improvement in computational efficiency,” but no one yet knows how to get AI to discover a new architecture on the level of the transformer; what is missing is smarter models. He even predicted that society may need to agree for a period of time to limit the IQ of a single model, “and if future progress is insufficient, that will stem from this decision, not from AI's ability to do research.”

He laid out his deflationary faith: the current five-year total cost of ownership for each GW of leading AI compute is about $70 billion, and it would be better if that could fall to $20 billion, while models would also need to improve performance another 10 times each year. The path to achieving this is to make energy cheaper and compress the stacked margins between “sand and output tokens.”

Regarding companies' anxiety about token budgets, he believes the more common problem today is not using AI too much, but not using it enough.

How should personal agents be priced?

Regarding the recently hot personal agent products (such as Muse), Altman's current judgment is that “the Macintosh moment has not yet arrived.” People like these products, but they have not yet reached the point of “I can't imagine life without it.” The winning form will be like current products, but “it'll just work”—reliable and able to handle more complex tasks. As for the cat-and-mouse game of computer use, he believes it will ultimately converge to the point where “you have the right to let AI operate your own computer on your behalf.”

On how agents will be commercialized, Altman believes companies are clinging to the AI pop-ups and HTML inside their own products, afraid of losing the customer relationship, but consumers want to have AI do everything on their behalf inside their main chat app. He believes the real moat comes from brand, product selection, and fulfillment networks, not HTML. He volunteered that he chooses Amazon Pay 10 times out of 10, even if the price is higher, and praised Amazon's fulfillment network, while acknowledging that the partner platforms inside ChatGPT are not good enough yet. He also called all web forms “fake work” and compared the present to the App Store era back then.

It is also for this reason that, when designing pricing, OpenAI almost followed AWS in quoting by task and completely abandoning token-based billing, but customers strongly opposed it: “I need to know what I'm paying for,” even if they pay more, they want tokens as an understandable unit.