DeepSeek did the math: $38 per GPU-hour — can the inference business make money?

On October 20, when DeepSeek open-sourced its inference framework DeepInfer, it announced a very eye-catching number: a single cluster can process 155 trillion Tokens per day, and if calculated according to DeepSeek API's off-peak pricing, each GPU hour corresponds to an average revenue of 38 US dollars.

This is a theoretical billing revenue converted based on Token processing volume and public API prices; the actual situation is far more complex. But this number is still too attractive: if roughly extrapolated linearly, calculated at 24 hours a day and 365 days a year, the annualized billing output value of one GPU exceeds 330,000 US dollars.

Over the past few years, the AI industry has been discussing how much money it takes to train a frontier model, how expensive Nvidia's GPUs are, and how much cloud vendors have invested in building data centers. But after these expensive computing devices are purchased, how much monetizable intelligence can they actually produce? Setting aside the dazzling "financial instruments," how much money can companies really earn?

DeepSeek's 38 US dollars provides a rare observation window. At a time when the capital market is once again questioning the returns on massive AI investment, feedback from the industry frontline is very optimistic. "Insiders" and "outsiders" have developed a huge divergence.

An entrepreneur running a Token relay business told Tencent Technology that in recent months, AI demand from small and medium-sized enterprises is "surging across the board," and he can feel this change from customers' actual usage. In July and August this year, only a few companies were trying things that peers are now commonly doing. Some enterprises that originally had data security concerns have also begun private deployment of models. He is also seriously considering "expanding production."

Another entrepreneur in the AI application field is also very clear in attitude: "I will still keep buying Nvidia stock." The B-end enterprises he contacts are accelerating the introduction of Agents, and from daily operations to customer service, more and more businesses are beginning to adopt AI.

But he also judges that the China-US AI markets may experience a "small trough" in the short term. Previously, news of model breakthroughs, productivity improvements, and AI replacing jobs continuously pushed up market expectations. When the capital market begins to demand that AI deliver results in real money, the actual outcomes of enterprise applications may not yet have caught up, and market enthusiasm will also decline accordingly.

DeepSeek's calculation provides another signal worth noting: even at off-peak prices, a highly optimized inference system can still achieve considerable unit GPU billing output value.

So, as inference efficiency continues to improve, if there are enough users continuously consuming Tokens, can AI already become a profitable business? The answer may be more optimistic than imagined, but also more complex.

One GPU, 38 US dollars per hour

In V4.1 Flash released on September 10, DeepSeek had already disclosed many technical details of inference efficiency optimization.

According to the official technical report, this model has 552 billion total parameters, adopts a mixture-of-experts (MoE) architecture and a new Causal Encoder–Decoder design, activates about 8 billion parameters when processing input, and activates about 16 billion parameters when generating output.

The model optimizes the cache structure for long context and Agent scenarios. According to the paper, global KV Cache usage is reduced to about one quarter of the previous generation, and long-term retained cache usage is reduced to about one eighth. The global KV Cache per Token is about 890 bytes, supporting up to 1 million Tokens of context.

These optimizations directly affect memory usage, data transmission, and service costs for long-context inference.

DeepInfer, open-sourced on October 20, further discloses how DeepSeek integrates these technologies into a large-scale online inference service.

It integrates previously open-sourced components such as FlashMLA, DeepGEMM, DeepEP, and TileKernels, while introducing mechanisms such as expert parallelism, speculative decoding, and asynchronous scheduling.

Figure: Schematic diagram of DeepSeek's online inference system

In its official deployment Q&A, DeepSeek highlighted three engineering principles: adapting to accelerator hardware from different vendors, deploying Prefill and Decode separately, and retaining KV Cache for as long as possible to reduce repeated computation.

In a production environment, Prefill can be deployed on ordinary 8-GPU servers, while Decode uses 64 to 128 GPU supernodes to improve throughput through large-scale expert parallelism. DeepSeek also proposes retaining KV Cache for more than 72 hours to meet the needs of Agents repeatedly calling the same context.

The core goal of these optimizations is to enable the same number of GPUs to serve more requests while meeting latency requirements.

However, this open-sourcing does not mean that DeepSeek has disclosed all components in the complete production environment.

The official README document points out that the KV Cache system, visual encoding and Prefill/Decode separated service orchestration, and load balancing across data parallel replicas still need to be supplied externally. Some components are deeply coupled with DeepSeek's internal infrastructure, so they have not yet been fully open-sourced.

DeepInfer is currently also mainly aimed at DeepSeek-V4.1 Flash. If other models differ greatly in MoE expert organization, attention structure, numerical precision, or hardware configuration, it is difficult to copy it directly.

Therefore, 38 US dollars does not mean that any enterprise can download DeepInfer and deploy a batch of GPUs to obtain the same revenue performance. DeepSeek has also not fully disclosed the cluster hardware composition, cost bills, paid traffic proportion, and revenue recognition method required to calculate this metric.

But it at least shows that under specific model and hardware conditions, inference engineering optimization can significantly increase the potential billing output value of computing devices.

Selling Tokens at low prices and generating relatively high billing revenue from one GPU are not contradictory.

The key question is how many effective tokens can actually be processed within an hour, and how many tokens can genuinely be billed to customers.

If inference can make money, why are model companies still losing money?

If inference efficiency has already reached this level, why do some large model companies still need to keep raising funds, and even keep losing money?

One important reason is that training frontier models and selling inference services are inherently two different sets of accounts.

In 2026, Stanford University's "AI Index" cited data from the research organization Epoch AI to estimate the computing expenditures of OpenAI and Anthropic in model R&D and inference services.

In 2025, OpenAI's computing expenditure was about 16.3 billion USD, of which R&D-related computing expenditure was 8.3 billion USD and inference expenditure was 8 billion USD, with the two accounting for almost half each.

In 2024, OpenAI's corresponding expenditures were 4 billion USD and 1.8 billion USD respectively. That is, within one year, inference computing expenditure grew by about 3.4 times, while R&D computing expenditure grew by about onefold.

Anthropic's computing expenditure in 2025 is estimated at 6.8 billion USD, of which R&D-related expenditure is 4.1 billion USD and inference expenditure is 2.7 billion USD. R&D still accounts for a relatively large proportion.

These figures show that the computing investment structure of frontier model companies is changing. As commercial users increase, inference has become an increasingly important source of cost, but R&D spending has not disappeared because of the expansion of the inference business.

A model company pursuing SOTA needs to bear the costs of continuous training and large-scale inference services at the same time. The former includes pretraining, post-training, reinforcement learning, data processing, model evaluation, and a large number of experiments, while the latter requires maintaining the computing power, capacity, and stability of online services.

Epoch AI specifically points out that R&D computing expenditure is not all used for the final formal training. A large amount of resources is actually consumed in experiments, research, and the model iteration process. Therefore, the publicly disclosed single-training cost of a certain model cannot simply be regarded as the entire R&D cost of a frontier model company.

Image source: Stanford HAI / Epoch AI, 2026

Another layer of pressure brought by commercialization is that the more users there are, the higher the inference cost may also be.

"The Information" reported in February this year that OpenAI's gross margin in 2025 was about 33%, lower than 40% in 2024 and also lower than the previous forecast of 46%. Anthropic's expected gross margin for 2025 is about 40%. Although it is significantly improved from the negative gross margin state in 2024, it is still 10 percentage points lower than its earlier target.

The report pointed out that one important reason why the two companies' gross margins were lower than expected is that actual inference costs were higher than originally estimated.

This seems to contradict DeepSeek's 38 USD: on one side is continuously improving inference efficiency, while on the other side is gross margin eroded by inference costs.

In fact, they reflect two aspects of the same business.

How many tokens a unit of GPU can produce is supply efficiency; how many GPUs a company needs to invest to meet different customers' latency, context length, stability, and peak demand determines the actual operating cost.

Even if the cost per unit token decreases, an increase in heavy users, longer task execution time, and a rise in the number of model calls may also cause total inference expenditure to grow rapidly. Especially in products with a fixed monthly fee, the price paid by customers may not increase synchronously with token consumption. Think of Tibo, who racks his brains every day to post "Reset cards."

Image: OpenAI product and platform lead Tibo (Thibault Sottiaux) posted a series of updates about Reset on X on October 11, among which "Day 6 / Reset" received about 2.9 million views.

A model company's inference service can have positive gross margin, while the company as a whole may still lose money because of continuous R&D, infrastructure, and operating investment.

Conversely, if it does not bear frontier model training and specializes in operating inference services, the cost structure of the business is completely different.

Can you make money by not training models and only selling tokens?

They do not take training the strongest foundation models as their core business. Instead, they deploy open-weight models such as DeepSeek, Qwen, and Llama, and through multi-chip clusters, inference frameworks, model optimization, and scheduling systems, they sell API calls, dedicated inference instances, or private deployment services to developers and enterprises.

For these companies, the models themselves do not necessarily need to be developed from scratch. Costs mainly come from GPU procurement or leasing, data centers and electricity, networking and storage, inference system R&D, model adaptation, as well as sales, operations and maintenance, and customer support.

The three most critical variables are output value per unit of GPU, actual paid utilization rate, and comprehensive operating cost.

Assume that a full burdened cost per GPU hour for an inference service provider is 10 USD, and it can use the 38 USD disclosed by DeepSeek as a reference for full-load billed output value. Then in theory, as long as it achieves about 26.3% effective billed output, it can cover this 10 USD cost.

If effective billed output reaches 50%, revenue is equivalent to 19 USD per hour, and after deducting 10 USD, 9 USD remains.

Of course, this is only an assumption used to illustrate business sensitivity. The throughput, concurrency, and latency of actual systems have complex relationships, and revenue under low load does not necessarily change linearly with full-load performance. 10 USD is also not the real cost disclosed by DeepSeek, and the calculation result should not be treated as its actual gross margin.

But this calculation shows that when the efficiency of the inference system is high enough, commercial returns may have considerable room. The problem is that GPUs themselves are not cheap, and buying or leasing GPUs does not mean immediately having stable revenue.

CoreWeave provides a real case for comparison.

In the second quarter of 2026, CoreWeave revenue was $2.575 billion, up about 112% year over year, adjusted EBITDA reached $1.510 billion, and the margin was 59%.

But under US generally accepted accounting principles, the company still posted a net loss of $626 million for the quarter. The problem lies in the high asset and capital costs of the compute business. CoreWeave needs to invest heavily in purchasing GPUs and supporting equipment, and the related assets depreciate year by year with use; at the same time, the company relies on debt financing to support expansion and also bears high interest costs. In the second quarter of 2026, its depreciation and amortization expenses reached $1.393 billion, and net interest expense was as high as $640 million.

Depreciation reflects the cost of compute equipment already purchased being gradually recognized over its useful life, and interest is the price paid to finance these investments. Even if GPU leasing brings considerable revenue, these two burdens can still significantly compress profits.

CoreWeave's financial report also shows that high compute utilization and rapid revenue growth do not mean that operating compute is necessarily profitable.

More complicated still, inference compute is not a fully homogeneous market.

DeepSeek's ability to achieve relatively high output value per GPU through deep optimization does not mean that ordinary inference service providers can achieve the same performance with the same hardware and the same price.

Therefore, inference services may be a business with considerable gross margin potential, but they will not automatically become a high-profit business. Engineering capability affects the lower bound of cost, customer demand and pricing power determine the upper bound of revenue, and capital cost affects how much profit can ultimately be retained.

In addition, the inference market also faces continuous downward pricing pressure.

After reducing the cost per Token, DeepSeek is also passing part of the efficiency gains to users through lower API prices. Once other companies adopt more efficient algorithms and systems, they may also join price competition.

The gains created by improved inference efficiency may not ultimately all remain with the companies selling Tokens.

From Nvidia, TSMC, and SK Hynix to data centers and cloud service providers, AI infrastructure investment has already brought considerable revenue and profit to some upstream companies.

But for companies operating inference services, after the equipment is purchased, they still need to continuously find customers and tasks in order to continue generating revenue.

At present, Coding is one of the AI application markets with the clearest commercialization, but even in this field with relatively strong willingness to pay, profitability is not easy.

The Information disclosed that in the quarter ended January 2026, Cursor's gross margin was once -23%; if the inference costs of free users were fully included in cost of sales, the gross margin was about -31%. Although the gross margin has since turned positive, the high cost of model calls remains a problem that AI applications need to face.

The situation for consumer applications is even more divided. On October 12, Character.AI, once a star AI companion company, filed for bankruptcy protection. According to bankruptcy filings and public reports, the company still had about 12 million monthly active users at the time, and revenue reached $5.3 million in August 2026, but monthly operating losses were still $8 million to $9 million.

In the first eight months of 2026 alone, Character.AI spent $56.1 million on compute, accounting for about 45% of operating expenses. Even after shifting to cheaper GPUs and open-weight models and cutting personnel expenses, it still failed to achieve profitability. Legal proceedings and regulatory pressure also worsened the company's difficulties.

However, AI-native applications are not incapable of making a profit. Midjourney founder David Holz said in an interview with The Information in March this year that the company's revenue had significantly exceeded $200 million in 2023, has continued to grow since then, and has remained profitable.

But it is also not a universal sample for all AI applications. Image generation has relatively clear use cases and payment value, while a personal Agent running around the clock may need to continuously call models, use tools, and replan tasks, making the cost structure more complex.

From the overall market perspective, demand is in fact growing rapidly.

The State of Consumer AI report released by Menlo Ventures in September 2026 estimates that global consumer AI spending has reached $40 billion, more than three times the $12 billion in 2025.

But it is worth noting that AI usage among US adults only rose from 61% in 2025 to 64%. The rapid growth in consumer AI spending mainly comes from existing users using it more deeply and paying more. New users are limited.

Image: Menlo Ventures consumer AI spending chart

Menlo Ventures partner Amy Wu Martin pointed out that the surprising change this year is that the consumer AI market achieved triple growth without welcoming a large number of new users. What drove the growth was heavy users bringing AI into more life scenarios and beginning to allow AI to take actions on their behalf.

Now, what matters more is how many tasks a user hands to AI each day, and how much actual value each task can create. This is also one of the reasons why Personal Agent (PA) has once again become a direction of common focus for major companies and startups.

Recently launched Duet, Dots, Muse, Grokbot, and a batch of products from independent startups, although they differ in entry points, memory, permission systems, and execution methods, are all trying to expand the range of personal tasks that AI can undertake.

If Agents can continuously handle emails, arrange work, operate software, complete procurement, and coordinate different services, AI calls may shift from short sessions initiated actively by users to long-process execution centered on actual tasks.

Collaboration among Agents, as well as more and more tool calls and service requests initiated directly by machines, may also further expand the inference consumption market.

However, more Token consumption itself does not represent commercial success.

If a PA company charges users $20 per month but needs to spend $30 on model calls and other services, then it may help upstream model providers expand revenue while still being unable to make a profit itself.

Similarly, increasingly efficient models may also make tasks that previously required large numbers of Tokens cheaper. Even if the number of Agents grows substantially, total market revenue may not necessarily grow at the same proportion.

Model companies hope to expand the inference consumption market, application companies hope to complete tasks at lower inference cost, and users care about how much the actual work completed is worth.

DeepSeek's $38 shows the potential revenue capacity of highly efficient Token production under specific conditions.

Over the past few years, the capital market has invested enormous amounts of money in AI, and chips, storage, and compute infrastructure have been the first to benefit. As inference systems continue to advance, model companies and application companies have been facing another more complex business proposition: if more and more companies can produce intelligence at lower cost, who can find enough customers willing to buy intelligence?

There is also a harder question: if AI is a new era, what increment should the "intelligence" carried by tokens create? The core logic of AI Native is still waiting in the midst of chaos.

Source of this article: Tencent Technology Risk Disclosure and Disclaimer

Markets carry risk, and investment requires caution. This article does not constitute personal investment advice and does not take into account the special investment objectives, financial conditions, or needs of individual users. Users should consider whether any opinions, views, or conclusions in this article are suitable for their particular circumstances. Investment based on this is at the user's own risk.