VentureBeat survey: 79% of enterprises use less than half of their GPU capacity, yet 70% still plan to expand computing power

Recently, continuous survey data for the third quarter of 2026 compiled by VentureBeat Intelligence, the research arm of tech media outlet VentureBeat, shows that 79% of enterprise respondents said their GPU compute capacity sits completely idle at least half the time.

This figure remained stable at a high level of 77% to 81% across all three months. In other words, among these surveyed enterprises, underutilization of GPUs has become a persistent phenomenon.

But even more puzzling is that among these enterprises whose average GPU utilization does not exceed 50%, 70% of companies still plan to introduce, expand, or replace new AI computing platforms within the next 12 months.

VentureBeat Intelligence, the research arm of VentureBeat, aggregated the results of three rounds of VB Pulse surveys on artificial intelligence infrastructure and compute conducted in July, August, and September 2026. The three rounds of surveys drew a total of 385 qualified respondents from organizations with no fewer than 100 employees, including consultants responding on behalf of their primary clients.

Now looking at return on investment. VentureBeat's September survey this year shows that among 64 respondents planning to adopt a new AI platform, 58 had no ROI data reviewed by the finance department, accounting for as much as 91%.

Of course, an unreviewed ROI does not mean the project has no returns. In the same survey, respondents without plans to adopt a new platform also generally lacked such data, but it at least indicates that the financial evaluation of enterprise AI investment may not have kept pace with the speed of infrastructure spending. That is to say, AI capabilities are rapidly entering production environments, but exactly how much money was spent and how much value was created remain questions many enterprises have not fully answered.

These numbers matter because Wall Street is fiercely debating whether AI has already formed a bubble, and based on its survey, VentureBeat has put forward a rather controversial judgment: the risk of an AI bubble may already have emerged inside enterprises...

Two mismatched sets of bills: idle "compute" and money-burning "APIs"

On the surface, this phenomenon of "idling on one hand while doubling down on the other" is extremely contrary to business common sense, especially at a time when most technology leaders are facing pressure to cut AI budgets: some technology leaders told VentureBeat that in this autumn's budget season, they are under enormous pressure to cut AI bills.

Professionals and consumers are using more and more AI, and much of that work relies on frontier models provided by Anthropic and OpenAI; this demand will continue to support these companies' compute investment for some time to come.

Panic-driven compute stockpiling: amid anxiety over the global compute shortage of the past two years, many enterprises, driven by a fear of "missing out," locked in and purchased GPU clusters in advance. However, once the hardware arrived, enterprises lacked mature internal use cases and engineering capabilities to run it at full capacity.

Path dependence on closed-source frontier models: business teams and engineers prefer to directly call the most advanced closed-source frontier models from OpenAI, Anthropic, and Google. Enterprises pay for these companies' APIs by token, or purchase model capabilities through monthly subscriptions or large cloud service providers, and cannot directly deploy them on the local or hosted GPUs they previously controlled.

This has created a highly ironic situation: self-built compute has become a stranded asset, while the cost of calling closed-source APIs is devouring budgets at an alarming rate.

In August this year, Fortune magazine reported that Uber's chief technology officer revealed that because the company encouraged employees to use AI tools as much as possible, it even set up a "leaderboard" ranking software engineers by their use of these tools.

As a result, the entire 2026 AI budget was exhausted within a few months, and the company had to declare the end of the "Tokenmaxxing" era.

The survey data further corroborates this structural divide: among enterprises planning to purchase new computing platforms and with idle GPUs, 90% are looking to public cloud and model API service providers, while only 8% are considering expanding local private clusters. At the same time, 91% of enterprises planning to introduce new platforms cannot produce a single actual ROI assessment reviewed by the finance department.

Of course, some compute idleness may be intentional on the part of enterprises, such as reserving GPU resources to cope with demand peaks, and the new platforms they plan to introduce may also be used to replace old platforms.

From "compute arms race" to "inference economics"

If idle GPUs expose the problem of infrastructure utilization efficiency, then the spread of Agents may leave enterprises facing even more complex cost challenges.

VentureBeat expects that the share of enterprise AI spending used for model calls will grow increasingly higher. In particular, as business shifts from simple generative dialogue to multi-step autonomous AI Agents, the compute contradiction facing enterprises is being further amplified.

Gartner predicts that by 2028, the inference cost of a single Agent workflow will rise more than 5 times.

This does not mean that the per-Token price of models will become increasingly expensive. In fact, as inference infrastructure and model efficiency continue to optimize, the unit price per Token may continue to decline, but the real problem is that the number of Tokens consumed by an Agent when completing a task may grow faster than the unit price declines.

One cost source that is easily overlooked is context. Take AI programming as an example: the model not only needs to understand the current problem, but also needs to continuously read project code, system instructions, and historical execution records. As the task progresses, this context may appear repeatedly across multiple requests. If it is reprocessed every time, costs will accumulate rapidly, which is also why context caching is receiving increasing attention.

In a technical guide, NVIDIA engineers mentioned that in the Claude Code session scenarios they studied, after the first call, about 85% to 97% of the context in subsequent requests can be reused through caching, while Anthropic's cache read price, depending on the model and pricing conditions, can be as low as one-fortieth to one-tenth of the price of ordinary input Tokens.

For Agents that need to continuously execute tasks, this optimization may bring considerable savings.

Another variable is the number of Agents. A programming task test by industry evaluation agency Vals AI found that the approach of using a main Agent to schedule multiple sub-Agents cost 1.8 to 5.1 times that of a single Agent.

However, among the four test configurations, only one achieved a statistically significant performance improvement. In other words, adding Agents may bring stronger capabilities, or it may simply cause more models to consume Tokens at the same time.

Farewell to extensive expansion: the path of pragmatic self-rescue for enterprises

Faced with an extremely high idle rate and out-of-control model invocation bills, for enterprises, extensive "blind procurement" may be becoming a thing of the past. The industry generally believes that before enterprises sign their next compute contract, they should prioritize squeezing efficiency out of existing assets: find ways to use the compute and Tokens they have already paid for to complete more actual work.

Specifically, the following things need to be done well:

Measure the cost of each completed task, while considering both GPU compute resources and API invocation spending;

Test different models and inference intensities to determine the enterprise's default configuration: the evaluation criterion should be the cost per completed task, not the price per individual Token. Because a cheap model may consume more Tokens, ultimately making the cost per task higher instead;

Provide Agents only with the context they actually need, and cache reused content: this should be done whether through model vendor APIs or when running on the enterprise's own infrastructure;

Add a second Agent only when the task can be split into parts that can be executed in parallel;

Migrate routine, high-frequency tasks from frontier model APIs to open models running on idle compute, such as classification, information extraction, and text summarization;

Use remaining GPU idle time to process tasks that can be deferred, such as batch inference, model evaluation, and fine-tuning.

The ultimate goal is for enterprises to pay a more reasonable cost for each piece of work actually completed.

This also gives new meaning to the surprising set of data at the beginning: 79% low GPU utilization and 70% willingness to adopt new platforms cannot directly prove that an AI bubble has formed, nor can they show that the entire AI industry is heading toward overcapacity. But it reminds enterprises that there may indeed be a gap between resource investment and actual demand that needs to be re-examined.

In the past few years, the AI industry has been accustomed to measuring progress by parameter scale, training compute, and capital expenditure. As technology gradually enters real business, beyond these metrics, a more pragmatic evaluation system is also needed: how much work have enterprises actually completed with AI? How high is the success rate? How much time has been saved? And how much cost has been paid?

For model vendors, this means competition will not only revolve around model capabilities, but will also extend to inference efficiency, deployment flexibility, and overall task cost. For enterprises, more important than continuing to hoard compute may be to first find ways to make good use of the GPUs and Tokens they already have.

After all, GPUs are bought to do work, and what is truly worth paying attention to is how much value they ultimately create...

Reference links:

This article comes from the WeChat public account "Machine Heart" (ID: almosthuman2014), author: one who follows AI, published by 36Kr with authorization.