Breaking down AI infrastructure from first principles: the multiple constraints of power, cooling, packaging, and capital

A server rack the size of a refrigerator, weighing more than a pickup truck; by 2027, its peak power will be equivalent to 65 households drawing electricity at the same time; to keep it from burning up, each rack needs to dissipate as much heat as 30 gas boilers.

This seemingly science-fiction statement is in fact the International Energy Agency (IEA)'s description of next-generation AI data centers in an April 2026 report. The report gives another set of figures: in 2025, the capital expenditure of five tech giants exceeded $400 billion, and in 2026 it will rise by another 75%—an amount that already exceeds total annual global investment in oil and gas extraction.

Over the past two years, "the end of compute is electricity" has become almost an industry consensus. That statement is correct, but only half correct. Electricity is of course a necessary condition, but what truly holds back AI expansion is often not just "whether there is electricity," but an entire chain strung together by power generation, grids, transformers, cooling, storage, packaging and capital—and the vastly different construction speeds of each link in that chain.

To see this chain clearly, the best approach is to temporarily set aside ready-made narratives such as the "GPU arms race" and "power anxiety," return to the most basic physical facts, and calculate the account from scratch.

How is intelligence "burned" into existence?

If all commercial packaging is stripped away, from a physical perspective what AI does is actually this: send electrical energy into transistors, drive bit flips, complete matrix multiplications, and finally spit out tokens.

Broken down this way, the "effective intelligence" a country or a company can produce can be roughly written as the product of four factors:

Available electricity × hardware energy efficiency (how many operations can be completed per joule) × algorithmic efficiency (how many useful tokens can be obtained per operation) × utilization rate (the proportion of time the machine is actually working)

This formula is not precise, but its advantage is that each factor corresponds to a specific layer of infrastructure. Electricity corresponds to power plants and grids, hardware energy efficiency to chips, memory, packaging and interconnect, algorithmic efficiency to model architecture and inference software, and utilization rate to scheduling systems, cooling and the reliability of power supply.

If any one factor fails, the other three will be dragged down no matter how well they are done. Precisely because of this, the bottleneck of AI infrastructure keeps "migrating": two years ago it was that GPUs could not be bought, later it was that advanced packaging and high-bandwidth memory could not secure production capacity, and now it is transformers, gas turbines and grid-connection queues.

First, look at two physical facts.

The first is conservation of energy. Apart from a very small portion that becomes optical signals and mechanical vibration, almost all the electricity a data center takes in ultimately becomes heat. This means "power supply" and "heat dissipation" are essentially two sides of the same problem: however many kilowatts you send into a rack, you must remove that many kilowatts of heat from it. As will be seen later, the surge in rack power density is simultaneously crushing traditional power distribution and air-cooling systems.

The second is that "moving data is more expensive than computing data." In chip design there is a classic set of calculations: under the 45nm process, one 32-bit floating-point addition consumes about 0.9 picojoules, while reading data of the same length from off-chip DRAM consumes hundreds of picojoules—a gap of two to three orders of magnitude.

After process advances the absolute values have changed, but this order-of-magnitude relationship has not fundamentally changed. In other words, the bulk of the energy consumption of modern AI chips is often not in "computing" but in "moving." From high-bandwidth memory (HBM) and CoWoS advanced packaging to NVLink high-speed interconnect, co-packaged optics and "super nodes," the underlying logic of almost all hardware innovation over the past few years has been the same thing: shortening the distance data has to travel.

So where is the physical limit? According to Landauer's principle, the minimum energy required to erase one bit at room temperature is about 3×10⁻²¹ joules. Taking Nvidia's Rubin GPU as an example, with roughly 50 PFLOPS of FP4 inference compute per package and about 2 kilowatts of power consumption, a rough conversion puts each FP4 operation at the level of tens of femtojoules. Even if one operation is broken down into hundreds of bit operations, today's chips still have multiple orders of magnitude of room from the theoretical lower bound (rough estimate, for order-of-magnitude reference only).

A tweet by Ming-Chi Kuo, analyst at TF International Securities in Hong Kong

So the energy problem of AI is not "hitting a physical wall," but a problem of engineering, supply chains and institutions. This judgment is important because it determines where we should look—after all, the real constraints are in factory scheduling tables, grid plans and the meeting minutes of local councils.

Demand: efficiency improves tenfold every year, so why is more and more electricity being used?

First look at the other end of the ledger: demand.

The IEA's latest estimates show that in 2025 global data center electricity consumption grew 17%, among which AI-focused data centers' electricity consumption surged 50%, while global total electricity consumption grew only 3% over the same period. Under its base case, data center electricity consumption will double from 485 TWh in 2025 to 950 TWh in 2030, accounting for about 3% of global electricity consumption; electricity consumption of AI-specific data centers will triple over the same period.

Efficiency is soaring, yet the total is soaring too—this is the classic Jevons paradox. The IEA breaks it down into three forces accelerating simultaneously: efficiency gains, surging usage, and new use cases unlocked by capability leaps.

Video generation, reasoning models, and agentic tasks can consume hundreds to thousands of times more energy per request than simple text generation. When AI shifts from "answering a question" to "spending half an hour completing a task for you," the compute consumption per interaction is no longer on the same order of magnitude.

The training side is expanding in tandem. According to statistics from research institute Epoch AI, the training compute of frontier models has grown about 4 to 5 times per year since 2020. After accounting for improvements in hardware energy efficiency and longer training durations, the power required for a single frontier training run is still growing at about 2.2 times per year, and the largest training runs currently exceed 100 megawatts; if the trend continues, a single training run could require 4 to 16 gigawatts by 2030.

Another Epoch tracking effort shows that the total compute of the global installed base of AI chips is growing at about 3.4 times per year.

China's data offers another footnote on the inference side. Yu Ying, deputy director of the National Data Administration, disclosed in May 2026 that as of the end of March, China's average daily token calls had exceeded 140 trillion, more than a thousandfold increase from the end of 2024; in early 2024, that figure was only about 100 billion. Compute supply is expanding in step: as of the end of July 2026, the country's total intelligent computing power reached 2.45 million PFLOPS.

It is worth noting that the 3% global share does not look alarming. The real pressure comes from concentration. In its 2025 report, the IEA estimates that by 2030, U.S. data centers will contribute nearly half of the growth in U.S. electricity demand, and the electricity used by U.S. data centers will exceed the combined total of all energy-intensive industries such as aluminum, steel, cement, and chemicals.

Electricity: When demand that iterates quarterly collides with a grid planned in decades

If you ask only "is there enough electricity in the world," the answer is yes. The problem is speed.

The construction cycle for AI data centers is shrinking dramatically. Epoch AI's observation is that gigawatt-scale data centers can be built in about two years; the IEA says two to three years. The rhythm of the power system is completely different: delivery lead times for large power transformers in the U.S. have stretched to as long as four years, gas turbine delivery times exceed three years, and new nuclear power is measured in decades.

The deepest contradiction in AI infrastructure is the mismatch of three clocks: chips iterate yearly, data centers are built in two years, and grids are planned over ten years. The IEA report says: the contrast between the speed of the AI revolution and the speed of the physical, social, and economic systems that support it is growing ever starker.

English text is being prepared.

A simple division shows the existing backlog is roughly equivalent to nearly six years of output. With large units booked out beyond 2028, some developers have turned to "second-best" options such as reciprocating internal combustion engine units.

The transformer story is more subtle. It accounts for a small share of total data center costs, yet it may determine whether an entire project can be powered on schedule. PwC analysts estimate delivery lead times for large power transformers in the U.S. at up to four years; further upstream, the grain-oriented electrical steel needed for transformer cores has only one domestic producer in the U.S., Cleveland-Cliffs. A supply chain scaled for "loads that basically do not grow for twenty years" suddenly has to serve a customer that raises its forecasts every quarter. According to Bloomberg, of the roughly 12 gigawatts of U.S. data center capacity originally scheduled to come online in 2026, only about one-third is actually under construction.

Facing grid interconnection queues, the mainstream choice for U.S. developers is "bring your own power." IEA satellite tracking shows that about one-fifth of a batch of U.S. data center on-site gas generation projects have begun site clearing or construction; by 2030, on-site gas could provide 15 to 27 gigawatts of power to data centers. But this path is not cheap: AI loads fluctuate sharply, and to provide stable power with on-site gas, generation capacity needs to be oversized by 30% to 70% relative to actual demand.

Nuclear power is a longer-term bet. Microsoft signed a 20-year power purchase agreement with Constellation to restart Three Mile Island Unit 1 (now renamed the Crane Clean Energy Center), with power generation to resume as early as 2027; Meta signed with Constellation to purchase 1.1 gigawatts of power from the Clinton nuclear plant starting in mid-2027; Amazon expanded its agreement with Talen to lock in up to 1.9 gigawatts of nuclear power. According to third-party tracking, the major cloud providers have signed about 9.8 gigawatts of nuclear-related agreements.

But the IEA cautions that the first small modular reactors (SMRs) are not expected to come online until around 2030. In other words, nuclear power solves the problems of the 2030s, not today's problems.

The power shortage will ultimately be transmitted through prices. PJM, the largest electricity market in the United States, covering 13 states and Washington, D.C., saw its capacity market clearing price jump from $28.92 per megawatt-day for the 2024/25 delivery year to $329.17 for 2026/27. PJM's independent market monitor estimates that about 40% of the $16.4 billion total cost of the December 2025 auction was driven by data center load. By the July 2026 auction for the 2028/29 delivery year, the price had hit the cap for the third consecutive time. Electricity is turning from an engineering problem into a livelihood and political problem.

Heat and density: racks are turning into small power plants

Back to the physical fact from the first section: to shorten the distance data must travel, GPUs need to be packed as close together as possible to form larger high-speed interconnect domains. The larger the interconnect domain, the more chips are packed into a single rack, and the higher the power density. This is a physical logic with almost no way back.

Look at the numbers. According to industry reviews, rack power in the Hopper era was about 40 kilowatts, rising to 120 to 140 kilowatts in the Blackwell era. A supply chain survey by TF International analyst Ming-Chi Kuo shows that Nvidia's soon-to-ramp Vera Rubin NVL72 rack offers two power configurations, with whole-rack power consumption of about 190 kilowatts and 230 kilowatts, both significantly higher than the roughly 140 kilowatts of GB300 NVL72; the 2027 Rubin Ultra "Kyber" rack is widely expected to reach about 600 kilowatts. According to IEA statistics, the power density of AI servers increased 11-fold between 2020 and 2025 and will quadruple again by 2027.

Nvidia's official blog description of Vera Rubin NVL72 reads more like an introduction to an industrial device than to a computer: the whole rack integrates about 1.3 million components and nearly 1,300 chips, weighs about 4,000 pounds, roughly equivalent to a pickup truck.

The surge in density is simultaneously breaking through both the power supply and cooling systems.

On the power supply side, Kuo points out that when rack power approaches 200 kilowatts, traditional 54-volt power distribution will encounter hard constraints in busbars, cable space and conversion efficiency, so Vera Rubin NVL72 may be the last generation of the Oberon rack architecture. The underlying physics is simple: transmitted power equals voltage multiplied by current, and line losses are proportional to the square of the current, so raising the voltage allows more power to be carried through thinner copper. This is exactly why the industry is moving to 800-volt high-voltage DC distribution, and the IEA also notes that AI data centers are borrowing the technology direction of the EV and new energy industries toward high-voltage DC.

On the cooling side, hundreds of kilowatts of heat far exceed the limits of air cooling, and full liquid cooling is shifting from optional to default.

There is also a new problem that is often overlooked: load fluctuation. The IEA found that server loads in AI data centers can swing repeatedly by more than 50% of rated capacity within a single second. Thousands of GPUs switch synchronously between the compute phase and the communication phase of training, which to the grid is like a giant switch being flipped constantly. As a result, energy storage is becoming standard equipment for AI data centers, and it is expected that by 2030 there may be 20 to 25 gigawatts of battery storage inside data centers worldwide; with the right incentives, they could instead become a regulating resource for the grid.

A rather symbolic detail is that Kuo disclosed that Vera Rubin offers two power configurations, "Max Q" and "Max P," to adapt to the power conditions of different data centers. Chip specifications are beginning to compromise with the power reality of machine rooms, which has been rare in the past. Data centers are turning from "buildings that house servers" into a kind of power electronic device, and chip companies are being forced to become power engineering companies as well.

Chips: why the bottleneck has moved from GPUs to memory and packaging

The reason still comes back to "moving data costs more than computing data." In large-model inference, especially the decoding stage where tokens are generated one by one, the bottleneck usually lies in how fast model weights and context caches can be moved out of memory—this is the so-called "memory wall."

The solution is to stack DRAM chips vertically into HBM, then use advanced packaging such as CoWoS to place it together with the GPU on the same interposer, shortening the data path from centimeters to millimeters.

The cost of this technological path is dragging the entire memory industry into shortage. In its April 2026 report, the IEA noted that the HBM shortage formed rapidly over the preceding six months and is expected to last at least until the end of 2027. SK Hynix said as early as October 2025 that its entire 2026 capacity had already been booked out; market research firm TrendForce estimated in early 2026 that average DRAM prices would rise 50% to 55% quarter-on-quarter in the first quarter. Because the same wafer capacity yields far fewer bits when producing HBM than ordinary memory, AI demand for HBM is directly squeezing memory supply for phones and PCs. According to IDC analysis, memory accounts for about 15% to 20% of the bill of materials for mid-range phones, and this "strategic wafer reallocation" is reversing the years-long trend of spec downgrading in the consumer electronics industry.

Advanced packaging is another chokepoint. By one estimate, TSMC's CoWoS monthly capacity will reach about 130,000 wafers by the end of 2026 and is expected to double to about 260,000 wafers by 2028; at the end of 2024, that figure was only about 35,000 wafers.

Even with nearly fourfold capacity expansion over two years, capacity remains fully booked. Morgan Stanley estimates that Nvidia occupies about 60% of CoWoS capacity. Ming-Chi Kuo estimates that the Rubin platform alone will require 300,000 to 350,000 CoWoS wafers in 2026.

From first principles, this shift is inevitable. After the slowdown of Moore's Law, increasing transistor density on a single chip has become ever harder, and performance gains increasingly come from "packing more silicon and memory into one package, then connecting more packages into one interconnect domain." As a result, the manufacturing bottleneck has shifted from front-end lithography to memory stacking and advanced packaging—once considered "back-end processes." Next, TSMC is advancing CoPoS, a larger-area panel-level packaging technology, while optical interconnects are moving from between racks to beside the chip package. Every step is still about shortening distance.

Money: $700 billion a year—who is paying?

What strings electricity, heat, and chips together is capital.

Statistics from several institutions differ slightly in methodology, but the direction is the same: the combined capital expenditure of Amazon, Alphabet, Microsoft, and Meta in 2026 is close to $700 billion. The IEA's judgment is that data center investment has grown so large that corporate balance sheets can no longer support it independently, and it must rely heavily on capital market financing; this means the pace of AI electricity demand growth will be highly sensitive to market sentiment, return-on-investment expectations, and the macroeconomic financing environment.

Cash flow pressure is already showing. CNBC reported that Alphabet's capital expenditure in the first half of 2026 rose 111% year-on-year to $98 billion, and its free cash flow turned negative for the first time in the second quarter; analysts expect Microsoft's free cash flow to turn negative in the fourth quarter as well.

A question more worth asking is: how much of the "gigawatts" on paper has actually been powered up? Take the Stargate plan led by OpenAI as an example. This project, which claims $500 billion in investment and 10 gigawatts of planned capacity, has its flagship site in Abilene, Texas. Estimates by Epoch AI based on satellite imagery show that, as of March 2026, the Abilene site had about 0.3 gigawatts of operating capacity, against a planned full load of 1.2 gigawatts. Other reports, citing OpenAI's chief financial officer, say that by the end of 2025 OpenAI's total effective compute capacity across all sites and partners was about 1.9 gigawatts.

"Signed," "under construction," and "powered up" are three completely different states, and most of the grand numbers in the industry refer to the first. Confusing the three creates both false prosperity and false collapse narratives.

Another detail reflects the changing place of electricity in the cost structure. According to reports, the CEO of Crusoe, the developer in Abilene, once said that including supporting gas-fired power plants, its data center costs about $19.2 billion per gigawatt, roughly twice the $9 billion to $11 billion per gigawatt for a traditional grid-connected "power shell." If this figure is accurate, then under the self-built power supply model, electricity is no longer an ancillary cost of data centers but a major cost item on par with chips.

Social license: the slowest constraint

Beyond electricity, heat, chips, and money, there is another constraint that cannot be directly solved by engineering means: whether local communities are willing.

According to tracking firm Data Center Watch, in the first quarter of 2026, at least 75 data center projects in the United States were blocked or delayed due to local opposition, involving about $130 billion—a single-quarter scale comparable to all of 2025; the number of active opposition groups doubled from 396 at the end of 2025 to 833, covering 49 states. In the second quarter, at least 45 more projects worth about $68 billion were blocked, bringing the first-half total to nearly $200 billion. In the first six weeks of 2026, state legislatures across the United States introduced more than 300 data-center-related bills, with statewide moratorium proposals appearing in 14 states.

Residents' reasons for opposition are varied: rising electricity bills, water use, noise, land occupation, and emissions from gas-fired power plants brought by the growing number of "bring-your-own-power" projects. IEA research offers a relatively balanced perspective: in systems with excess power supply, stable new load can instead spread fixed grid costs and lower electricity prices; but data centers are built quickly, their load is uncertain, and grid-connection capacity is often overstated, so in regions with tight supply and demand they can indeed push prices up.

This constraint is the slowest precisely because it cannot be accelerated with orders and capital. Transformers can expand production, gas turbines can work overtime, but trust can only be rebuilt one community at a time. In this sense, whether data centers can transform from a "grid burden" into a "grid resource"—for example, by pairing with energy storage to participate in demand response and bearing grid upgrade costs—will likely determine the ceiling of the next round of expansion.

Extreme extrapolation: sending data centers into space?

If first principles are pushed to the limit, a natural question arises: since the constraints come from power, heat dissipation, and land, why not go to a place where the sun never sets and there are no neighbors to complain?

This is exactly the logic of orbital data centers. In November 2025, Google announced the "Suncatcher" plan, envisioning the deployment of TPU-equipped satellites in dawn-dusk sun-synchronous orbit. Google says that in a suitable orbit, solar panels can achieve power generation efficiency up to 8 times that on the ground, and it is reportedly in talks with SpaceX on launch cooperation.

Yesterday, Google successfully performed inference using an in-orbit TPU and wrote "a haiku about being in space." SpaceX went further, applying to the U.S. Federal Communications Commission in January 2026 to deploy up to 1 million data center satellites; startup Starcloud already sent an H100 into orbit in 2025. In China, the "Three-Body Computing Constellation" has completed the launch of its first 12 computing satellites and deployed multiple AI models in orbit; Alibaba's Qwen3 was deployed to a space computing center by Guoxing Aerospace's "Star Computing Plan," completing end-to-end inference in orbit.

But physics also sets obstacles for this approach. There is no convection in a vacuum, so heat can only be dissipated by radiation, requiring enormous radiator panels; frontier model training requires thousands of chips to synchronize gradients at every step, and the bandwidth requirements far exceed the capabilities of current inter-satellite links; launch cost remains the decisive variable, and Google's own paper assumes that only by the mid-2030s might launch prices fall below $200 per kilogram.

A more accurate understanding may be this: space data centers are not an answer to "not enough energy," but an answer to "grid connection too slow" and "social license too difficult" on the ground. They remind us that the truly scarce commodity for AI infrastructure is energy that can be connected to computing quickly, legally, and reliably.

Conclusion

Back to the opening line: "The end of compute is electricity."

Starting from first principles, a more complete statement would be: as long as token demand grows faster than energy efficiency improves, the electricity line on the AI ledger will continue to grow; and what determines how fast a country or a company can run is the speed of its slowest link.

Looking back over the past three years, the path of bottleneck migration is clearly visible: first the GPU itself, then HBM and CoWoS advanced packaging, and now transformers, gas turbines, and grid-connection queues. What will be next? If the capital market's patience runs out before the infrastructure does, the bottleneck may become money; if community opposition continues to spread, the bottleneck may become permits; if the one-year-per-generation chip iteration continues to outpace the three-year construction cycle of data centers, the bottleneck may also become the stranded-asset risk of "a completed data center that cannot fit the next generation of chips."

No single technology can solve the problem once and for all. What is certain is that AI competition has evolved from a race over models and chips into a systems engineering competition covering energy, manufacturing, finance, and social governance. In this competition, what is most easily overestimated is a single-point breakthrough, and what is most easily underestimated are the inconspicuous links in the chain: a transformer, a piece of grain-oriented silicon steel, a planning permit.

This article comes from the WeChat public account "Machine Heart" (ID: almosthuman2014), author: someone focused on AI Infra, published by 36Kr with authorization.