SemiAnalysis Founder Analyzes AI Investment, Nvidia's Moat and the Memory Shortage

At CASPA's 35th Annual Conference held in the United States on October 23, SemiAnalysis founder and CEO Dylan Patel, Bernstein Managing Director and Senior Analyst Stacy Rasgon, and Austin Lyons discussed the outlook for the semiconductor industry, focusing on AI infrastructure investment, supply chain bottlenecks, and the chip competitive landscape.

Patel noted that demand for AI compute remains strong, but capacity constraints in key areas such as substrates and memory are limiting infrastructure expansion. At the same time, financing costs have risen sharply, making it increasingly difficult for emerging cloud service providers without long-term customer orders to obtain funding. Whether AI investment can continue to advance depends not only on demand but also on projects' financing capacity and returns.

Regarding Nvidia, Rasgon believes that performance improvements in its products can still support relatively high selling prices, but cloud providers' in-house chips are changing the competitive landscape. As in-house chip initiatives by Google TPU, Amazon Trainium, and companies such as OpenAI advance, Nvidia may face dual pressure on market share and pricing power even if it continues to grow.

The memory market faces another test: HBM demand is strong and industry profit margins are high, but manufacturers' capacity expansion may gradually change the supply-demand relationship after 2028. Over the next two years, financing capacity, supply of key components, and competition from in-house chips will become key variables affecting the pace of AI infrastructure expansion and the distribution of profits across the industry chain.

Wallstreetcn summarizes the key points as follows:

AI investment still has room to grow, but financing costs are becoming an important constraint. The U.S. 10-year Treasury yield has exceeded 5%, and the unsecured bond yields of some emerging cloud service providers are close to 15%. Financing is more difficult for companies lacking long-term customer orders, and project construction timelines will increasingly depend on financing capacity and investment returns.

Semiconductor supply chain bottlenecks will be difficult to resolve in the short term, with substrates and memory being key constraints. Substrates, HBM, optical interconnect components, and cleanroom capacity all face supply constraints. By comparison, power shortages can be partially alleviated through on-site power generation and other means, while expanding semiconductor supply chain capacity requires more time.

The memory shortage may persist until new capacity is released, and the supply-demand landscape faces a turning point after 2028. HBM demand is strong, and the memory industry's gross margin is said to have reached 88%–90%; however, if new capacity comes on stream in concentrated fashion after 2028 while demand growth slows, memory prices and profit margins may come under pressure.

Nvidia still has a performance advantage, but in-house chips are challenging its pricing power. Nvidia continues to support high selling prices through performance improvements, but in-house chip efforts by Google TPU, Amazon Trainium, and companies such as OpenAI provide large customers with alternatives. Even if Nvidia's revenue continues to grow, its market share and profit margins may not rise in tandem.

AI infrastructure costs continue to rise, and investment returns will become the focus of the next stage. The cost of building a 1-gigawatt compute data center has risen from about $40 billion in 2024 to more than $60 billion. Cloud providers are accelerating in-house chip development to lower cost per unit of compute, and whether enterprises can continue to expand the model of purchasing tokens through cloud services will affect the long-term returns on infrastructure investment.

The financing threshold for AI infrastructure construction is rising significantly. The U.S. 10-year Treasury yield has exceeded 5%, and the unsecured bond yields of emerging cloud service providers such as CoreWeave are close to 15%. The bond Volulta issued to build a cluster for Anthropic carried a 12% coupon but was sold at 95% of face value, for an actual yield of about 15%.

Rasgon noted that hyperscale cloud providers with abundant cash flow in the past, such as Meta and Google, are also consuming large amounts of free cash flow in AI infrastructure construction, and some companies may even need equity financing in the future to support investment.

Patel further noted that construction demand undertaken by non-hyperscale cloud providers accounts for nearly half of the total market, but such companies usually lack sufficient balance sheet support. Without long-term offtake agreements from hyperscale cloud providers, projects would be difficult to finance. To this end, Nvidia has begun supporting the construction of some compute clusters through arrangements such as revenue sharing, while Broadcom is providing chips to Anthropic in a similar installment-payment manner.

This means financing capacity is becoming an important dividing line for AI infrastructure expansion. Strong demand does not guarantee that all projects will be completed on schedule; for emerging cloud service providers with high capital costs and reliance on external financing, long-term orders and cash flow will directly affect the pace of construction.

Substrates and memory are harder to expand quickly than power

On supply chain bottlenecks, Patel listed substrates as one of the most critical constraints at present. Relevant Japanese suppliers are relatively cautious about capacity expansion, limiting the output increase of accelerated chips, CPUs, and other products.

Memory is another major bottleneck. Rasgon noted that producing the same capacity of HBM requires about 3–4 times as many wafers as DDR5, so growing AI chip demand has intensified competition for memory capacity. At the same time, components needed for optical interconnection, such as lasers and fiber connections, are also in short supply; even if CPO mass production accelerates around 2028, related supply may still struggle to meet demand.

The wafer fabrication equipment market is likewise constrained by cleanroom capacity. Rasgon believes that the forecast of the global wafer fabrication equipment (WFE) market exceeding $200 billion in 2026 is overly optimistic. By raising utilization at existing fabs, the market size could rise from about $120 billion expected at the start of the year to about $155 billion, but new cleanrooms will only come online gradually in 2027, with more capacity expected to be released in 2028–2029.

By contrast, there are more ways to address power shortages. Patel expects that in 2027, data centers under construction worldwide will total nearly 50 gigawatts, of which about 35 gigawatts will be in the United States. On-site gas turbines, reciprocating engines, and fuel cells can alleviate power constraints to some extent.

When will the memory shortage end? Supply released after 2028 is key

The memory industry's current high profits reflect the imbalance between AI demand and limited capacity. Rasgon said memory industry gross margins have reached 88%–90%, but the market has not therefore given related stocks extremely high valuations; one reason is precisely that investors worry new supply will be released in a concentrated wave after 2028, causing prices to fall.

What is special about HBM is that its manufacturing consumes more wafer resources. As demand for AI accelerators grows, expanding HBM production not only requires adding its own capacity but also affects the supply of other DRAM products. In the short term, capacity allocation and the pace of expansion will struggle to keep up quickly with demand; but as new capacity gradually comes online, supply and demand could also reverse.

When the memory shortage ends depends not on a fixed date, but on when new capacity begins to exceed incremental demand. If expansion is concentrated after 2028 while AI demand growth slows at the same time, current high prices and high margins may be difficult to sustain.

Investors need to watch HBM and DRAM expansion progress, advanced packaging and wafer manufacturing capacity, and changes in cloud providers' capital expenditure. Before supply increases markedly, memory may still constrain AI compute delivery; but over a longer cycle, expansion could also become a factor that pushes down prices and erodes profits.

Can Nvidia's moat hold? Performance advantage faces challenge from in-house chips

Nvidia's core advantage lies in continuously improving chip performance and strengthening customer stickiness through its software ecosystem and developer tools. But as cloud providers place increasing emphasis on compute costs, its high prices also create room for in-house chips to develop.

Rasgon noted that Nvidia's valuation relative to the Philadelphia Semiconductor Index is already at a historical low, reflecting market concerns about the sustainability of AI investment and competition from in-house chips.

In terms of product iteration, Nvidia is still supporting pricing through performance improvements. From Ampere to Hopper, average selling prices roughly doubled and performance improved about 10x; from Hopper to Blackwell, prices rose about 50% and performance improvements are said to reach 3–30x. As long as performance gains continue to exceed price increases, customers will be motivated to pay more for new-generation products.

But competitors are accelerating their catch-up. Patel's research shows that Nvidia's share of demand for related coprocessor wafers and HBM has begun to decline. At the same time, Google TPU has been sold to external customers, Amazon Trainium continues to improve support for the PyTorch ecosystem, and OpenAI and Anthropic are also advancing in-house chip plans.

These efforts may not immediately replace Nvidia, but they can provide large customers with alternatives and strengthen their bargaining power. For cloud providers with massive compute demand, the appeal of in-house chips lies not only in lowering procurement costs but also in reducing dependence on a single supplier.

Enterprise AI adoption: buying tokens is more realistic than building compute in-house

On the enterprise application side, the three analysts believe most enterprises will not purchase large amounts of hardware themselves to run private models, but will instead buy tokens through cloud services—that is, call cloud-based AI inference capabilities and then combine them with their own data and business processes. This is similar to the past trend of enterprise IT shifting from self-built infrastructure to cloud computing.

Currently, AI-assisted programming is one of the more mature enterprise applications. Rasgon shared that with Claude, a financial model that originally took a week to complete was built in only about 1 hour and 20 minutes, involving 10 years of historical data, multiple business restructurings, and logical inference from management earnings call content.

The value of such applications is first reflected in saving time and reducing repetitive work, rather than necessarily showing up immediately in revenue. Its return on investment still needs to be verified through actual business, but enterprises calling models through the cloud also means that demand for AI infrastructure will increasingly depend on usage of inference services.

If enterprise applications continue to expand, cloud providers will be motivated to keep investing in compute; but if construction and financing costs rise faster than revenue, the returns on infrastructure investment will still come under pressure.

The following is the full text:

Host Austin Lyons:

Let's start from a more macro perspective, first discussing demand, then supply. On the demand side, AI revenue is growing very fast, but the growth rate has begun to slow. At present, revenue across the entire AI industry combined is still only a small part of total industry spending, so the market is scrutinizing every revenue data point extremely carefully.

On the supply side, compute capacity for the next several years has already been booked out, and prices remain firm. From the perspective of a semiconductor analyst, how should we understand these two signals? And where will value in the industry chain ultimately flow?

Today, we have invited two important figures from the semiconductor industry: Dylan Patel is the founder and CEO of SemiAnalysis, one of the most influential research firms in the AI infrastructure space; Stacy Rasgon is a managing director and senior analyst at Bernstein who has spent more than a decade analyzing semiconductors on Wall Street. I am today's host, Austin Lyons, currently chief analyst at Chief Strat and formerly a principal engineer at Intel.

Today, we will discuss the future of the semiconductor industry, looking ahead to 2027 and 2028, covering areas including financing, GPUs, CPUs, and wafer foundries.

Host Austin Lyons:

Recently, the cost of capital has become a focal point of market discussion. Hyperscalers are pouring in enormous amounts of money, and free cash flow has turned negative. At the same time, the U.S. 10-year Treasury yield has broken above 5%, and bond market yields for companies such as CoreWeave are approaching 15%.

Should we be worried about this? As companies continue to ramp up investment and financing costs keep rising, how big an impact will this have on AI infrastructure buildout? Let's start with Stacy.

Stacy Rasgon:

Looking at stock performance, share prices of the relevant companies are still up considerably from before, but they are clearly below their highs. The market is sending a signal: investors are worried that there is great uncertainty about the pace and sustainability of infrastructure buildout, and that industry growth may be approaching a peak. Financing is clearly one important issue, perhaps even one of the most important.

As you just mentioned, hyperscalers were once among the strongest cash-generating businesses in human history, but now they are burning through a lot of cash and may need to issue equity in the future. Meta recently adjusted its equity structure, one purpose of which was to provide room for this kind of financing arrangement. Companies such as Google have also raised financing.

Take CoreWeave as an example: its bond yield is close to 15%. It is important to note that this does not represent the actual financing cost of all the company's current debt, but rather the market trading yield on its unsecured bonds. But if the company re-entered the market today to issue unsecured bonds, its financing cost would likely be around this level. That is a fairly high cost.

Still, I think that for at least the next year or two, the industry can broadly keep operating. More importantly, the relevant revenue must continue to grow in order to gradually ease market concerns about financing.

Take the compute rental market as an example: supply is extremely tight right now, and the common refrain we hear is that no one can get enough compute. Some companies rent out compute through short-term contracts, and by some estimates, revenue per gigawatt of compute could even reach around $50 billion.

So it is not hard to imagine a scenario in which, even though construction costs are high, the relevant revenue is still enough to cover costs, or even exceed them. But the question is: how long can that level of revenue last? Ultimately it still depends on how strong demand is relative to supply, and how long the supply-demand gap can persist.

These questions may require Dylan to make a deeper judgment from the perspective of industrial supply and demand. But financing is undoubtedly one of the many constraints right now.

One more point worth noting: constraints slowing growth is not necessarily entirely a bad thing. It may prevent the industry from expanding too much and avoid investment spiraling out of control.

Something similar happened during the internet bubble. At the time, a large amount of dark fiber was laid, that is, fiber that had been laid but not yet put into use, and much infrastructure was ultimately overbuilt, with some of it still not fully utilized today.

Today's construction constraints may instead allow supply and demand to match more reasonably and enable industry growth to last longer. If today there were no constraints at all and expansion proceeded frantically, the future might instead face a more severe risk of collapse.

So I don't think all constraints are bad. Of course, how the market views these constraints is another matter, and they have clearly affected the short-term performance of the relevant stocks.

Dylan Patel:

I agree with Stacy. Looking back at infrastructure buildouts over the past few decades, every time the industry expanded, companies often double-ordered, or even ordered two or three times as much, ultimately building too much capacity. The supply chain has never been good enough at forecasting demand.

Now, financing is playing a certain self-regulating role. A year or two ago, as long as a cloud service provider was willing to pay a high enough interest rate, it could usually get funding; but now, without a long-term offtake agreement of sufficiently high credit quality, some emerging cloud service providers cannot build compute clusters.

Hyperscalers can of course push free cash flow down to near zero, and they can keep borrowing, and their profits are still growing, so their buildout capacity will increase further. But they cannot invest indefinitely. Next year, they cannot raise capital expenditure to more than $2 trillion, or even $2.5 trillion. Funding capacity will ultimately limit the scale of spending.

Now look at neoclouds. Non-hyperscale cloud providers account for nearly half of the buildout demand in the entire market. If they do not have offtake commitments from hyperscalers, how are they supposed to obtain financing?

Today, even with procurement commitments from customers such as Anthropic or OpenAI, the market may demand a yield of 13%–15% on the related debt. Voluta is one example: the company is building a compute cluster for Anthropic, and its bonds carry a coupon of about 12%, but they were issued at only 95% of face value, so the effective financing yield is close to 15%.

If a company does not have procurement commitments from hyperscalers or large AI labs, how can it build compute to serve small and medium-sized businesses and startups? Frankly, without the support of these large customers, financing would be very difficult.

Nvidia has already realized that this is a major problem. There are a handful of large customers in the market that can buy as much compute as their own funds allow; but there is also the other half of market demand—how can those customers be served?

Nvidia has therefore begun to adopt some innovative financing structures. For example, in projects such as Sharon AI and Fermis, Nvidia supports project financing by sharing revenue from the compute cluster. The arrangement is roughly as follows: Nvidia leases the cluster at a price that basically guarantees the cloud service provider makes little profit, and if the cloud service provider leases the compute to other customers at a higher price, Nvidia then shares in part of the revenue.

Similar arrangements are increasing. When Lambda built a compute cluster for Anthropic, it also had to sign a revenue-sharing agreement with Nvidia, with a sharing ratio of about 10%.

This shows that whether emerging cloud service providers can obtain enough capital, complete construction, and ultimately even expand to the point of oversupply is constrained by financing conditions.

We also see that the industry's financing methods are shifting from long-term loans and long-term debt to more flexible arrangements. When Broadcom supplied chips to Anthropic, it adopted a lease-to-purchase-like approach, extending the payment period over several years.

For companies that cannot rely on their own balance sheets for financing the way hyperscalers do, obtaining compute requires companies such as Nvidia, Broadcom, or AMD to provide some financing support or credit backing for the cluster.

Therefore, financing is an important factor limiting the expansion of AI infrastructure. Hyperscalers themselves also have enormous on- and off-balance-sheet commitments. For example, Meta's commitments for computing and data centers are reportedly already at $1.4 trillion.

These projects can be funded in a variety of ways: companies can borrow to buy data centers, or lease data centers or related equipment from other companies, and the party providing the infrastructure then raises financing on that basis. But it is becoming increasingly difficult for the entire industry to continue expanding substantially on this basis.

Capital markets are even beginning to feel uneasy about Meta's financial arrangements. Meta's own financial condition does not mean it cannot bear these investments, but a commitment scale this large is already enough to put the market on alert.

Ultimately, AI infrastructure construction has three interrelated core conditions: land, power, and shell facilities; chips; and financing. A constraint in any one link could slow overall expansion.

II. Where are the bottlenecks in the semiconductor supply chain in 2027–2028?

Host Austin Lyons:

The financing situation is now fairly clear: hyperscalers are relatively able to bear the investment, but emerging cloud service providers face greater pressure, and the industry is also trying new financing structures.

If we set financing aside for now and return to traditional supply chain bottlenecks, which links will actually constrain industry growth over the next two years?

ASML says its 2028 capacity is already sold out; TSMC's demand also consistently exceeds supply, even though it is expanding 2nm process capacity. The market also widely believes that power is an important bottleneck.

Dylan, you recently mentioned that nearly 50 gigawatts of data centers may come online next year. Could you rank the various links in the semiconductor supply chain by degree of constraint?

Dylan Patel:

Within the semiconductor supply chain, apart from memory, I think the biggest bottleneck is substrates, including packaging substrates, PCB materials, and other related materials.

These links have long been underinvested. Many related materials suppliers in Japan operate in a relatively conservative style and are unwilling to expand production on a large scale, so over the next two years, whether AI accelerators, CPUs, or other components in the supply chain, all will be significantly constrained.

In my view, substrates are one of the most severe bottlenecks right now, with a degree of constraint even exceeding land, power, data center shell facilities, and some semiconductor manufacturing equipment.

The second-biggest bottleneck is memory.

The third category of bottleneck is optical interconnect, including co-packaged optics (CPO) and near-packaged optics (NPO). It is not easy for the related industry to significantly increase output. Even if all companies want to adopt CPO in 2028, or a single large company wants to deploy it fully by then, they will not be able to obtain enough lasers, fiber connection components, and other supporting parts.

As for land, power, and data center shell facilities, power is indeed an important constraint. But our research finds that the market may be overestimating the constraints facing data center construction next year.

Nearly 50 gigawatts of data centers worldwide are expected to be under construction next year, of which about 35 gigawatts are in the United States, with the rest distributed in regions such as Australia, India, and Malaysia. In fact, data centers are being built rapidly all over the world.

Although power remains tight, companies can take a number of steps to ease the constraints, such as deploying gas turbines, reciprocating engines, or fuel cells on-site at data centers.

So although constraints on power and data center construction conditions do exist, their impact may be smaller than that of substrates, memory, lasers, and other optical supply chain components.

Power supply can be increased in a relatively short period through some fairly straightforward measures, whereas the semiconductor supply chain involves far more complex manufacturing processes and material systems, and expanding capacity takes longer.

Stacy Rasgon:

That judgment is very interesting. Both Nvidia and Broadcom have given quite high growth guidance. Jensen Huang has said Nvidia's revenue is expected to grow at least 70%; Broadcom's growth forecast is even 100%. But these guidance figures are themselves already constrained, and the constraints companies are mainly considering seem to be land, power, and data center construction conditions.

Perhaps this is because they are semiconductor companies themselves and have a clearer idea of how many chips can be produced, so they focus more on data center construction.

I agree that there are some solutions on the power and land side. However, even on-site power generation may not be achievable quickly. How long does it actually take to order a gas turbine these days? Musk even said he was prepared to manufacture turbine blades himself to bypass supply constraints.

The memory market is especially worth watching. Memory stocks are currently valued very low, with many trading at forward P/E ratios of only two or three times, or three or four times, even though memory prices have already risen sharply.

I am not directly responsible for memory stock research, but take SanDisk as an example: its quarterly earnings per share guidance is even higher than the company's stock price about two years ago. This shows how strong the current price increases are.

However, memory stocks generally trade at single-digit P/E ratios, because the market usually prices stocks this way when the industry cycle is near its peak. Investors worry that, especially after 2028, once new supply begins to be released, memory prices will fall sharply.

I keep thinking: what if that does not happen?

Look inside an AI rack: about 70%–80% of the silicon area is actually memory, mainly DRAM. High-bandwidth memory (HBM) has another special problem: producing the same capacity of HBM requires roughly three to four times as many wafers as ordinary DDR5 DRAM.

This means HBM production will consume a large amount of wafer resources, squeezing out capacity that could otherwise be used to produce ordinary memory needed for PCs, smartphones, and other products.

III. Memory and Manufacturing Equipment: The Shortage May Last Until 2028

Stacy Rasgon:

Demand for HBM is very large, which leads to another bottleneck: semiconductor manufacturing equipment, that is, the equipment used to produce these chips.

This week, one company released a preliminary outlook for next year's wafer fab equipment (WFE) market, forecasting a market size of more than $200 billion. I found this forecast quite surprising, because I think it is very difficult for the market size to reach a higher level, and even whether this figure can be achieved is doubtful. The reason is simple: we do not have enough fabs.

Cleanrooms are almost entirely occupied. Companies have been maximizing utilization of existing facilities as much as possible, which is why at the beginning of the year the market estimated WFE at about $120 billion, while this year it may ultimately reach $155 billion. The growth mainly comes from more full use of existing facilities, rather than large-scale new plant construction.

But there is ultimately a ceiling to the utilization of existing facilities. Companies must build new cleanrooms, and that takes years. New cleanrooms will begin to come online in small amounts in 2027 and increase further in 2028 and 2029. Only then will we be able to judge more clearly whether supply and demand will become unbalanced.

If demand continues to grow at the pace currently expected by the market, memory may remain in a state of tight supply for a long time.

Substrates, PCBs, and other materials have similar problems. Equipment suppliers have said one after another that short-term supply constraints have already affected their ability to increase output. What is constrained is not only PCBs and semiconductor chips, but also seemingly ordinary components such as valves and mass flow controllers. Shortages of these basic parts will also affect other parts of the supply chain.

During the pandemic, semiconductor equipment companies and most companies in the supply chain learned some lessons, and they now manage supply issues more maturely than in the past. But many links are still constrained, and these problems must be resolved first before capacity constraints in other links can be eased.

So the supply chain actually has layer upon layer of constraints. I think the shortage situation will not disappear in the short term. This year is not a big problem, but next year may still be tight, and 2028 will most likely not be loose either. As for 2029, it will depend on demand at that time. Overall, I expect tight supply to persist for some time.

IV. Pricing Power in the Semiconductor Industry Is Changing

Host Austin Lyons:

If supply remains tight in the coming years, and the memory industry's margins have already reached more than 80%, or even 88%–90%, will companies across the entire supply chain continue to raise prices and earn higher profits?

Stacy Rasgon:

The semiconductor industry has a very interesting characteristic: in the past, people almost took continuously falling prices for granted. In fact, the semiconductor industry was historically an industry that severely destroyed economic value, unable to cover its cost of capital for a long time, with most of the value it created flowing to consumers.

Many years ago, when I worked at McKinsey, I participated in a study on value creation in the semiconductor industry. We used the classic definition of economic profit, comparing a company's return on invested capital with its weighted average cost of capital, and aggregated the results across the entire industry. The study covered about 150 companies, 12 sub-segments, and a full industry cycle.

The initial time frame of the study was roughly 1996 to 2004. The results showed that the industry as a whole cumulatively destroyed about $5 billion in economic profit. If Intel is excluded, the scale of destruction reaches about $50 billion. At the time, that was a rather startling figure.

In other words, the value created by the industry was not enough to cover the cost of capital, and economic value in effect flowed to consumers. That was an era when buying semiconductor products was very cost-effective.

After 2008, the industry underwent consolidation and concentration, and companies began to gradually improve profitability, at least slowly becoming able to cover the cost of capital.

The pandemic changed the industry's understanding of its own pricing power. Companies discovered that they actually had strong pricing power. Previously, the industry had not truly realized this.

During the pandemic, input costs such as raw materials kept rising, but not one of the many companies I tracked saw its gross margin decline because of rising costs. They all raised prices, and by enough to offset the cost increases.

For example, a company with an original gross margin of 60% could, if costs rose by 100 units, raise its price by 160 units, thereby maintaining its original gross margin. Customers also accepted these price increases.

I believe the pandemic-era inflation actually helped the semiconductor industry improve its profitability. In 2023 and 2024, industry shipments were roughly flat compared with 2019, but overall average selling prices (ASP) were about 50% higher, even excluding memory.

Today, this pricing power is even more evident. Total semiconductor industry revenue is expected to reach about $1.5 trillion this year, or perhaps even higher, an increase of nearly 100% year on year. Of that, about 70%–80% of the revenue increment comes from higher memory average selling prices, and a considerable portion of the rest also comes from price increases in non-memory products.

By comparison, shipment growth is much smaller. The industry is exercising its pricing power more fully, and customers have generally accepted these prices, because when supply is tight, they often have little choice.

Therefore, as long as supply remains tight, I think this situation can continue.

Dylan Patel:

In the semiconductor industry, different companies use pricing power in different ways. Some companies raise prices sharply during booms but then suffer severely during downturns; others are able to maintain margins relatively steadily throughout the cycle.

Take TSMC as an example. Over the past decade, its gross margin has steadily risen from about 45% to about 60%. It can indeed raise prices during booms, but during downturns it has not given back much of the profit it previously gained.

The memory industry is different. Just about two years ago, the industry was in one of its worst downturns since the dot-com bubble, with gross margins even negative. Today, memory makers' profitability has improved dramatically.

Therefore, pricing power at different links in the supply chain depends on each one's market structure. TSMC has a near-monopoly position and maintains long-term relationships with customers; even though it has the ability to raise prices, it has not raised them as sharply as some other companies.

Memory makers are different. If customers such as Apple exert great pressure on suppliers during a downturn, then when the cycle reverses and demand exceeds supply, suppliers will also raise prices. What we are seeing now is exactly this kind of cyclical shift in bargaining power.

V. Compute Buildout Costs Are Soaring—Who Will Bear the High Costs?

Dylan Patel:

In 2024, the total cost of building 1 gigawatt of compute capacity was about $40 billion, covering chips, land, power, and data center facilities, among other things. Today, that cost has exceeded $60 billion.

Of course, not many companies actually built 1 gigawatt of compute capacity in 2024, but by 2026, more and more companies are starting to build compute facilities on that scale. At the same time, the computing power each gigawatt of capacity can provide is also increasing.

But the question is, such a gigawatt-scale compute cluster may be extremely profitable.

I mentioned earlier that cloud service providers often must obtain long-term purchase commitments from hyperscale cloud providers in order to finance a cluster. That means they need to lock in prices in advance, which also limits their room to earn excess profits.

Musk took a different approach: if chips, land, power, and data center facilities are all constrained, he can build the infrastructure first and sign lease contracts only once it is actually put into use. That way, when compute supply is very tight, he has a chance to rent it out at higher prices without having to lock in lower prices in advance.

At the same time, AI labs themselves can also earn huge revenue from inference services. According to relevant disclosures, Anthropic's revenue per gigawatt of inference compute could reach $70 billion–$80 billion.

But these revenues do not mean margins can stay high indefinitely. If a company obtains compute at a cost of $15 billion per gigawatt, margins could reach 75%–80%; if the cost rises to $50 billion, margins will fall sharply.

Therefore, who ultimately bears these costs remains an open question.

Nvidia must also face this pressure. Although its margins have remained broadly stable, or even declined slightly, it has had to absorb part of the cost increases in memory as well as in other supply chain links such as substrates and optical components.

The result is that the prices Nvidia charges customers have risen sharply, but its own margins have not risen by the same proportion.

This makes the distribution of profits across the supply chain especially important: who can protect their margins, who can expand them further, and who is forced to bear the pressure of rising costs? These questions are changing the industry's pricing landscape in real time.

In the future, customers may sign similar contracts at different prices. Companies with strong sales capabilities that can seize opportunities when supply and demand are tight may be able to obtain very high prices.

Host Austin Lyons:

This is very similar to the pandemic-era change Stacy just talked about: the semiconductor industry gradually realized that it can charge for the value it creates.

Dylan, you also mentioned that system suppliers like Nvidia must decide, when the prices of various components are rising, whether to pass costs on to customers or absorb the costs themselves and maintain margins.

The question is that infrastructure construction will clearly continue in the coming years, with limited supply and strong demand, but financing is becoming increasingly difficult. So will large cloud providers and model companies such as Meta one day conclude that compute has become too expensive to bear and begin to resist further price increases?

Stacy Rasgon:

When evaluating these investments, you cannot look only at costs and margins; you also have to look at what companies get after spending the money.

Take Nvidia as an example. The price of each generation of products is rising, but the performance improvement customers get is often even greater. This point is very important.

From Ampere to Hopper, the average GPU selling price roughly doubled, but performance improved by about 10 times. From Hopper to Blackwell, the situation is harder to compare directly because the two architectures differ considerably, but a rough estimate is that the average selling price per GPU rose by about 50%, while performance improved by 3–30 times, and with subsequent software optimization the actual improvement may be even greater.

Then from Blackwell to Rubin, rack costs may rise sharply again. According to reference data provided by Nvidia, a Blackwell rack costs about $25 billion, and a Rubin rack about $40 billion, but performance improves by about 10 times.

So the key is not that the construction cost per gigawatt of compute rises from $25 billion to $40 billion, or from $40 billion to $60 billion, but how many tokens and how much computing power customers actually get for every dollar they spend.

As long as the performance obtained per unit of cost continues to improve, this kind of investment remains attractive.

But if Nvidia's next generation of products improves performance by only 20%, I would start to worry a great deal about its pricing power and margins.

Nvidia's gross margin exceeds 70%, and it is built on a huge revenue base. The market keeps asking: in the face of competitors and other GPU suppliers, how can it continue to maintain such high margins?

My view is that as long as Nvidia can continue to make performance improvements significantly exceed average selling price increases, there is no need to worry excessively about margins. As product complexity rises and costs increase, prices must also rise; the key is the relationship between performance and cost, which must always benefit customers.

If the pace of innovation slows, I would be very worried.

I remember attending the Hot Chips conference around 2023. At the time, Nvidia chief scientist Bill Dally gave a keynote speech, and one point left a deep impression on me: over the past decade, GPU performance had improved by about 1,000 times. Nvidia achieved this progress through a variety of different technical means, and Dally said the goal for the next decade was to improve performance by another 1 million times.

If Nvidia can continue along this trajectory, I will be less worried about its margins. But if it cannot maintain the pace of innovation, or if other competitors quickly catch up, the situation will be completely different.

Ultimately, someone has to bear these costs. Ideally, customers are willing to pay for the value they receive. But if costs rise at an unsustainable rate, suppliers will have to start bearing part of them, and the fight over profits among different links in the supply chain will become more intense.

6. The Rise of Custom Chips: How Long Can Nvidia's Competitive Advantage Last?

Host Austin Lyons:

Dylan, more and more companies are starting to design their own chips. OpenAI's case is especially worth watching. Over the next two years, how should we view the development of GPUs and the rapid growth of various XPUs and AI accelerators? Quite a few startups are trying different chip architectures. How will this change the competitive landscape?

Dylan Patel:

More and more companies are trying to design their own chips, and this is already a clear trend. OpenAI demonstrated an important fact: even when a company is designing a chip for the first time, it can still produce a product with excellent performance.

OpenAI once invited us into its lab. We used SemiAnalysis's open-source inference benchmark InferenceX to test a variety of open-source models, and the results showed that its first-generation chip outperformed Nvidia's Blackwell.

This chip made extensive use of AI tools in the design and programming process. Anthropic has also begun advancing its own chip project, and hopes to complete development in half the time OpenAI took. These goals are quite aggressive, but if models can significantly speed up the design process, they are not entirely impossible.

Meanwhile, Google has already begun selling TPUs externally, and Amazon's Trainium has also become quite mature and can support PyTorch well. Meta and Microsoft likewise have very aggressive chip roadmaps.

There is a strong economic incentive behind this. We mentioned earlier that the cost of building 1 gigawatt of compute has risen from $40 billion to more than $60 billion. If the profits of chip design companies such as Nvidia, Broadcom, and Marvell could be eliminated, the cost of the same scale of compute might be only around $20 billion.

In other words, if the profit cost in the chip design segment can be significantly reduced, then in theory the money saved on 1 gigawatt of compute could support building more compute. For companies that need to run large numbers of inference tasks, this economic benefit is very obvious.

Therefore, Nvidia must maintain an extremely fast pace of innovation. It cannot merely double performance per dollar every year, or other competitors may quickly catch up.

If Nvidia can continue to sustain this pace of performance improvement, it of course has reason to continue earning high profits. But if large customers can design chips themselves, deploy them, and expand production scale, while obtaining memory, TSMC wafers, substrates, and optical components at prices close to supply-chain cost, then they have a lot of room to cut prices.

If the industrial chain further splits, and Nvidia or Broadcom no longer hold the same dominant position as in the past, the situation could change a great deal. This is also one reason the market worries that the valuation multiples of the two companies are at historic lows.

For example, will Meta reduce its dependence on Broadcom? Will Google do the same? Will these large customers eventually stop buying Nvidia chips and turn entirely to in-house designs?

At present, component suppliers themselves may also gain strong pricing power. The real question is whether Nvidia and Broadcom can continue to maintain their currently astonishing profit margins. For at least the next two years, I think they still have a chance to do so, but beyond two years it is hard to say.

Take our InferenceX benchmark as an example: getting an Nvidia inference system running properly took about six months; AMD took six to nine months; Google and Amazon each took only a few months; while OpenAI's chip took only about a month and a half.

The process from a chip being able to work to software being able to run on the chip is rapidly shortening. Now, companies are beginning to turn their attention to chip design itself.

Will chip design, like software development, gradually become a more widespread and standardized capability? If the answer is yes, the number of companies able to design chips could increase substantially.

Of course, chip design still requires paying fixed costs such as masks and electronic design automation (EDA) tools, and these inputs alone can reach $50 million. But if AI continues to lower the barriers to design and programming, the number of companies participating in chip design may still continue to grow.

Once chip design becomes more widespread, industry profit margins may decline. We have already seen some software-as-a-service (SaaS) companies suffer market revaluation; at the same time, Nvidia and Broadcom's earnings are still growing at more than double the rate each year, but their stock price performance has relatively stagnated.

This is precisely one of the most noteworthy questions for the future of the semiconductor industry.

7. Can XPU startups challenge GPUs? Nvidia's market share is changing

Host Austin Lyons:

So how will the GPU and XPU markets develop over the next two years? More and more AI accelerator startups are trying different architectures. Will the competitive landscape change significantly?

Dylan Patel:

For XPU companies, I think there are two different paths.

If you are a hyperscale cloud provider, you can take a relatively steady design route, make products similar to Google's or Nvidia's, and rely on your own scale to drive adoption.

But if you are a startup, you must try a very aggressive technology route that is clearly different from existing solutions. Looking back at the first generation of AI hardware startups, an important reason Cerebras and Groq were relatively successful is that they took bolder technology paths.

For the startups emerging now, I think only those whose technology solutions are unique enough that they cannot be easily copied by Nvidia, Google, or AMD truly have a chance to stand out.

The good news is that a large amount of capital is currently flowing into this field, and various aggressive architectures and design ideas are being tested.

At the same time, Nvidia's share of the industrial chain has already begun to change. From 2023 to 2024, Nvidia occupied a very high proportion of HBM and advanced packaging capacity; by 2025, this growth began to slow, and in 2026 its share will decline further. According to our judgment, this trend may continue next year.

But this does not mean Nvidia's profits will immediately decline. The entire market is growing too fast; even if its share falls, Nvidia's earnings next year may still double.

Google, Amazon, and AMD are all gaining share. You cannot look only at chip counts here, because chips vary greatly in size and value. More meaningful metrics include advanced packaging wafers, HBM usage, and spending on foundry and memory.

All of these metrics indicate that competitors are gradually expanding their market share.

Looking ahead to 2028 and 2029, Nvidia's growth rate may slow, and it may not be able to sustain annual revenue growth of 100%. At that point, the market will need to judge: will slower growth further lead to a decline in its share, or can Nvidia maintain its lead through a new-generation architecture after Rubin?

Nvidia must continue to accelerate the pace of innovation. Its one-generation-per-year chip update cadence is not only for its own development, but also to make it hard for competitors to catch up.

Nvidia's moat is not just hardware. CUDA software, its vast developer ecosystem, the infrastructure already deployed, and its own balance sheet and financing capacity all constitute competitive advantages.

Jensen Huang is clearly highly vigilant about competition. He has learned some lessons from Intel's history and always keeps an eye on competitors' moves.

Stacy Rasgon:

This also explains why Nvidia's valuation is at such a low level.

About nine to ten months ago, I wrote a research report comparing Nvidia's relative valuation with the entire semiconductor industry. At that time, over the past decade, Nvidia's relative valuation had been lower than that on only 13 trading days.

And now, its relative valuation is even lower than it was then.

But within the semiconductor industry, Nvidia's biggest threat clearly comes from competition. It is currently far ahead of its peers, but all competitors want to seize its market.

This is also a consistent feature of the semiconductor industry: as long as one company succeeds, there will always be people trying to challenge it. Nvidia must ensure it always stays ahead, and as the industry develops, this will only become harder.

8. Enterprise AI applications: Will more companies buy chips themselves in the future, or buy tokens directly?

Host Austin Lyons:

Earlier we talked about GPUs, XPUs, and competitors continuously expanding their market share. Our discussion mainly focused on data centers and hyperscale cloud providers.

So, for a company like Nvidia, is enterprise on-premises computing also an important growth opportunity?

Stacy Rasgon:

Nvidia now distinguishes revenue from hyperscale cloud providers and non-hyperscale cloud providers, but this division is not entirely accurate. Some AI labs may be classified as non-hyperscale cloud providers, while hyperscale cloud providers' spending may also indirectly support these labs, so there is overlap between the two.

Roughly speaking, the two types of customers currently each account for about half. Revenue contributed directly or indirectly by AI labs may account for about 20% of Nvidia's total revenue. That is still a fairly large proportion.

Therefore, Nvidia's revenue sources may be more diversified than we previously imagined.

It is still early stage, but I think at least one core application with clear value has emerged on the enterprise side: programming.

The value of enterprise applications may not necessarily be directly reflected in revenue growth; it may also be reflected in productivity gains, which are harder to measure.

I myself am a financial analyst and need to build financial models. A few months ago, our team tried using Claude to do this kind of work. The initial results were not ideal, but later a colleague thought the tool had improved significantly.

So we arranged a team meeting and had it build a financial model from scratch for a company we already cover.

The result was surprising: it took only one hour and twenty minutes to complete a model. The model covered ten years of historical data, handled multiple business restructurings and segment changes, and also attempted to use logical inference for forecasts based on what the CEO said on earnings calls, rather than simply generating numbers at random.

This work, if assigned to one analyst on the team, might take a week; if I did it myself, it might take three months. But Claude took only one hour and twenty minutes.

This means the same team can accomplish more work, or even accomplish more tasks with fewer people.

Based on actual experience so far, I increasingly believe that AI applications can deliver real returns. However, this is still an early market. Many enterprises are still in the experimentation stage, are not clear on how to use AI, and lack sufficient expertise to judge specific needs, so adoption will take time.

Overall, I am positive about what I have seen so far.

Dylan Patel:

There are obvious economies of scale in model inference. For most enterprises, a more reasonable approach is to buy tokens directly, that is, call cloud-based model inference services, and then combine these capabilities with their own data.

This is very similar to the development of enterprise IT over the past two decades. Twenty years ago, enterprises usually purchased infrastructure and deployed systems themselves; today, most IT spending has shifted to cloud computing. In the future, most enterprise spending on AI may also go toward buying tokens, rather than buying computing power themselves and running open-source models.

Even pharmaceutical companies and hedge funds may mainly pay to use models directly, rather than buying hardware themselves to run open-source models. Of course, there will still be some providers in the market that specifically procure computing power, deploy models, and provide services to customers.

There is still a lot in this discussion worth exploring in depth, but we are already over time. Today's conversation was very interesting, and thank you all for participating. Risk disclosure and disclaimer

Markets carry risk, and investment requires caution. This article does not constitute personal investment advice and does not take into account the specific investment objectives, financial situation, or needs of individual users. Users should consider whether any opinions, views, or conclusions in this article are suitable for their particular circumstances. Investment based on this is at the user's own risk.