VentureBeat survey: 79% of enterprises' GPUs sit idle at least half the time, yet seven in ten still plan to expand AI compute

Recently, continuous survey data for the third quarter of 2026 compiled by VentureBeat Intelligence, the research arm of tech media outlet VentureBeat, shows that 79% of enterprise respondents said their GPU compute capacity sits completely idle at least half the time.

This figure remained stable at a high level of 77% to 81% throughout the three months. In other words, among these surveyed enterprises, underutilization of GPUs has become a persistent phenomenon.

But even more puzzling is that among these enterprises whose average GPU utilization does not exceed 50%, 70% of companies still plan to introduce, expand, or replace new AI computing platforms within the next 12 months.

VentureBeat Intelligence, the research arm of VentureBeat, aggregated the results of three rounds of VB Pulse surveys on artificial intelligence infrastructure and compute conducted in July, August, and September 2026. The three rounds of surveys attracted a total of 385 qualified respondents from organizations with no fewer than 100 employees, including consultants responding on behalf of their main clients.

Now looking at return on investment. VentureBeat's September survey this year shows that among 64 respondents planning to adopt a new AI platform, 58 had no ROI data reviewed by the finance department, accounting for as much as 91%.

Of course, unaudited ROI does not mean the project has no returns. In the same survey, respondents without plans to adopt new platforms also generally lacked such data, but it at least indicates that the financial evaluation of enterprise AI investment may not have kept pace with the speed of infrastructure spending. That is to say, AI capabilities are rapidly entering production environments, but how much money was spent and how much value was created remain questions that many enterprises have not fully answered.

These numbers matter because Wall Street is fiercely debating whether AI has already formed a bubble, and VentureBeat, based on its survey, has put forward a rather controversial judgment: the risk of an AI bubble may already have appeared inside enterprises...

Two mismatched sets of bills: idle "compute" and money-burning "APIs"

On the surface, this phenomenon of "idling on one side while doubling down on the other" is extremely contrary to business common sense, especially at a time when most technology leaders are facing pressure to cut AI budgets: some technology leaders told VentureBeat that in this autumn's budget season, they are under enormous pressure to cut AI bills.

Professionals and consumers are using more and more AI, and much of that work relies on frontier models provided by Anthropic and OpenAI. This demand will continue to support these companies' investment in compute for some time to come.

Panic-driven compute stockpiling: Under the anxiety of the global compute shortage in the previous two years, a large number of enterprises locked in and purchased GPU clusters in advance out of a fear of "missing out." However, after the hardware arrived, enterprises lacked mature internal scenarios and engineering capabilities to run it at full capacity.

Path dependence on closed-source frontier models: Business teams and engineers prefer to directly call the top closed-source frontier models under OpenAI, Anthropic, and Google. Enterprises pay for these companies' APIs by token, or purchase model capabilities through monthly subscriptions or large cloud service providers, and cannot directly deploy them on the local or hosted GPUs they previously controlled.

This creates a highly ironic situation: self-built compute has become a stranded asset, while the cost of calling closed-source APIs is devouring budgets at an alarming rate.

In August this year, Fortune magazine reported that Uber's chief technology officer revealed that because the company encourages employees to use AI tools as much as possible, it even set up a "leaderboard" ranking software engineers based on their use of these tools.

As a result, the entire 2026 AI budget was exhausted within a few months, and the company had to declare the end of the "Tokenmaxxing" era.

The survey data further corroborates this structural divide: among enterprises planning to purchase new computing platforms and with idle GPUs, 90% are looking to public cloud and model API service providers, while only 8% are considering expanding local private clusters. At the same time, 91% of enterprises planning to introduce new platforms cannot produce any actual ROI assessment reviewed by the finance department.

Of course, some compute idleness may be intentional on the part of enterprises, such as reserving GPU resources to cope with demand peaks, and the new platforms they plan to introduce may also be used to replace old platforms.

From "compute arms race" to "inference economics"

If idle GPUs expose the problem of infrastructure utilization efficiency, then the spread of Agents may leave enterprises facing even more complex cost challenges.

VentureBeat expects that the share of enterprise AI spending used for model calls will become increasingly high. In particular, as business shifts from simple generative dialogue to multi-step autonomous AI Agents, the compute contradiction for enterprises is being further amplified.

Gartner predicts that by 2028, the inference cost of a single Agent workflow will rise more than 5 times.

This does not mean that the per-Token price of models will become increasingly expensive. In fact, as inference infrastructure and model efficiency continue to improve, the unit price per Token may continue to decline, but the real problem is that the number of Tokens consumed by an Agent to complete a task may grow faster than the unit price falls.

One cost source that is easily overlooked is context. Take AI programming as an example: the model not only needs to understand the current problem, but also needs to continuously read project code, system instructions, and historical execution records. As the task progresses, this context may appear repeatedly across multiple requests. If it is reprocessed every time, costs will accumulate rapidly, which is why context caching is receiving increasing attention.

In a technical guide, NVIDIA engineers mentioned that in the Claude Code session scenarios they studied, after the first call, about 85% to 97% of the context in subsequent requests can be reused through caching, while Anthropic's cache read pricing, depending on the model and pricing conditions, can be as low as one-fortieth to one-tenth of the price of regular input Tokens.

For Agents that need to continuously execute tasks, this optimization may bring considerable savings.

Another variable is the number of Agents. A programming task test by industry evaluation organization Vals AI found that the approach of using a main Agent to orchestrate multiple sub-Agents cost 1.8 to 5.1 times that of a single Agent.

However, among the four test configurations, only one achieved a statistically significant performance improvement. In other words, adding Agents may bring stronger capabilities, or it may simply cause more models to consume Tokens at the same time.

Farewell to extensive expansion: the path of pragmatic self-rescue for enterprises

Faced with extremely high idle rates and runaway model invocation bills, for enterprises, extensive "blind procurement" may be becoming a thing of the past. The industry generally believes that before enterprises sign their next compute contract, they should first squeeze efficiency out of existing assets: find ways to use the compute and Tokens they have already paid for to complete more actual work.

Specifically, the following things need to be done well:

Measure the cost of each completed task, while considering both GPU compute resources and API invocation spending;

Test different models and inference intensities to determine the enterprise's default configuration: the evaluation criterion should be the cost per completed task, not the price per individual Token. Because a cheap model may consume more Tokens, ultimately making the cost per task higher instead;

Provide Agents only with the context they actually need, and cache reused content: this should be done whether through model provider APIs or when running on the enterprise's own infrastructure;

Add a second Agent only when the task can be split into parts that can be executed in parallel;

Migrate routine, high-frequency tasks from frontier model APIs to open models running on idle compute, such as classification, information extraction, and text summarization;

Use remaining GPU idle time to process tasks that can be deferred, such as batch inference, model evaluation, and fine-tuning.

The ultimate goal is to enable enterprises to pay a more reasonable cost for each piece of work actually completed.

This also gives new meaning to the surprising set of data at the beginning: 79% low GPU utilization and 70% willingness to adopt new platforms cannot directly prove that an AI bubble has formed, nor can they show that the entire AI industry is heading toward overcapacity. But it reminds enterprises that there may indeed be a gap between resource investment and actual demand that needs to be re-examined.

In the past few years, the AI industry has been accustomed to measuring progress by parameter scale, training compute, and capital expenditure. As technology gradually enters real business, beyond these metrics, a more pragmatic evaluation system is also needed: How much work have enterprises actually completed with AI? How high is the success rate? How much time has been saved? And how much cost has been paid?

For model vendors, this means competition will not only revolve around model capabilities, but will also extend to inference efficiency, deployment flexibility, and overall task cost. For enterprises, more important than continuing to hoard compute may be to first find ways to make good use of the GPUs and Tokens they already have.

After all, GPUs are bought to do work, and what is truly worth paying attention to is how much value they ultimately create...

Source of this article: Machine Heart Risk Disclosure and Disclaimer

Markets carry risk, and investment requires caution. This article does not constitute personal investment advice, nor does it take into account the special investment objectives, financial conditions, or needs of individual users. Users should consider whether any opinions, views, or conclusions in this article are suitable for their particular circumstances. Investing based on this is at their own risk.