AI News

When tokens become a business, enterprises should do the math on AI

TMTPost · AGIUpdated Nov 16 19:06

AI summary

Lai Junjie, Vice President of Engineering and Solutions for NVIDIA China, proposed the four pillars of "Token Economics"—value, demand, supply, and monetization—noting that a single agent task may involve 50 to 500 consecutive model calls, and multi-agent collaboration further amplifies token consumption. China's average daily token calls have surged from 100 billion in early 2024 to 140 trillion in March this year, and enterprises are shifting their accounting from how many GPUs they bought to how much business value each AI investment returned.

Body · AI translation

Text | Light Cone Intelligence, Author | Luo Zhenhao, Editor | Liu Junhong

Over the past two years, the phrase heard most often inside enterprises has been "I want to get on AI," but what exactly AI is used for and how well it works is something many people have no sense of. Too many companies chose to "get it running first and figure it out later," until one day a sky-high Token bill suddenly surfaced, and only then did they realize this investment simply couldn't be accounted for.

But some companies have this account figured out down to the last detail: a supermarket chain giant used AI to negotiate supplier contracts, cutting costs by 3%, achieving a 4x ROI, and even extending payment terms by 35 days...

Over the past two years, the most visible change in the AI industry has been models getting bigger and compute getting stronger, but once the industry entered the agent and enterprise-application stage, companies began facing a more practical question: what exactly did the compute and Tokens they spent buy back?

This is also why "Token economics" has begun entering the industry's field of view.

According to disclosures from the National Data Administration: in early 2024, China's daily average Token calls stood at 100 billion; by the end of 2025, that figure had jumped to 100 trillion; by March this year, it had surpassed 140 trillion, a more than thousandfold increase in two years. Behind this set of data is the fact that the way AI is used is changing. In the past, companies were accustomed to calculating how many GPUs they had bought and how many data centers they had built; now what they calculate is how many tasks a given AI investment actually completed, how much time it saved, and how much revenue it created.

As AI moves from "generating content" to "continuously getting work done," Tokens are also shifting from a technical metric to a business metric.

In the past, Tokens were mostly just a unit of measurement for model calls; now, as agents begin performing multi-turn reasoning, calling tools, pulling in context, and even having multiple agents collaborate over long periods, the demand for Tokens has become more diversified.

In the view of Dr. Lai Junjie, Vice President of Engineering and Solutions for NVIDIA China, a single agent task may involve 50 to 500 consecutive model calls, and multi-agent collaboration will further amplify Token consumption. This means the AI account at enterprises is being recalculated.

Therefore, what truly deserves attention is no longer just how cheap Tokens are, but how the value of Tokens themselves is defined, how demand is forecast, how supply efficiency is improved, and ultimately how it all converts into business returns.

NVIDIA summarizes this as the four pillars of Token economics: value, demand, supply, and monetization.

From this perspective, what Token economics really discusses is a new set of operating logic for the AI industry after it moves from technology investment to productivity realization.

If, in the previous stage, the core question companies discussed around AI was how much compute to invest, then in this stage the real question is becoming whether the Tokens produced by that compute can form sustainably measurable business value.

Around this question, Light Cone Intelligence founder Wang Yisu and Dr. Lai Junjie, Vice President of Engineering and Solutions for NVIDIA China, recorded a podcast about "Token economics."

NVIDIA, standing on the AI infrastructure side, discusses how models, GPUs, networking, inference optimization, and Serving frameworks jointly determine the unit cost and delivery efficiency of Tokens. Light Cone Intelligence, as an industry observer, starts from the enterprise application angle and asks whether AI has actually entered real business, and how Token consumption, process efficiency, and final operating results can be put on the same spreadsheet.

The following is a transcript of the conversation, edited by Light Cone Intelligence without changing the original meaning:

Q: Please both of you, based on your own practical experience, talk about how to use Tokens more economically and efficiently?

Dr. Lai: Inside NVIDIA, we very much encourage employees to explore how to integrate AI into daily workflows, making AI an assistant that boosts personal efficiency. Our CEO, Mr. Jensen Huang, has also mentioned that in the future every employee will have their own AI assistant. All our internal teams—engineering, product, legal, marketing, finance, and so on—are constantly exploring how to integrate AI into specific workflows. From this angle, collaboration between humans and AI is becoming a new way of working.

From my personal experience, the key to using Tokens is not simply pursuing reduced consumption, but finding scenarios where AI can truly amplify individual capabilities, so that every AI interaction generates higher value. For example, in information organization, material analysis, problem decomposition, or other highly repetitive tasks, letting AI participate in more steps can help us reduce repetitive work, thereby freeing up more time for judgment and creative work.

Wang Yisu: We can't do without AI now. As content creators, our main spending is on in-depth content research and video production, and I personally have also tried some small AI Coding tools. Specifically, information lookup and deep research during daily writing are used the most, and our video team is also starting to use it more and more for material generation. In terms of spending, each person spends roughly a few hundred yuan per month; we use memberships for some overseas AI software and for mainstream high-quality domestic AI services.

Q: The same 1 million Tokens, used to call different models and applied to different tasks, may ultimately produce completely different value. Dr. Lai, could you first explain what a Token actually is?

We can understand it with a simple formula: the cost per unit of Token equals the data center's total hourly operating cost divided by the number of Tokens the data center delivers in real time per hour. This cost includes many aspects: GPU, CPU, networking, storage, power supply, and so on. When evaluating AI infrastructure investments, many enterprises tend to focus on only one of these costs, such as the price of GPUs. But what truly determines Token cost is often overall efficiency—how many usable Tokens the same investment can actually produce.

So from the perspective of Token economics, we need to pay attention to both the data center's total cost of ownership and its end-to-end output efficiency, shifting the evaluation standard from input metrics to output metrics.

Q: In other words, the value of a Token is not singular; it is related to the infrastructure behind it, model capabilities, application scenarios, and ultimately business results. So what are the key factors that specifically affect Token value?

Dr. Lai: To judge whether a Token has value, you can look at two core dimensions.

The first dimension is intelligence content. The same 1 million Tokens, generated by a frontier reasoning model with long-context capabilities versus generated by a lightweight model, carry completely different understanding, reasoning ability, and contextual information; the former has higher intelligence density.

The second dimension is interactivity, that is, the speed at which Tokens reach users. For scenarios such as coding assistants and high-frequency conversations, Tokens that arrive close to real time are valuable; for tasks such as batch document processing, Token latency is less sensitive.

But both dimensions must rest on one premise: value is relative to the use case. A small model post-trained for a specific scenario may achieve or even exceed the performance of a large model at only a fraction of the cost. The intelligence premium you pay extra is only worth paying when it can actually be used.

So when enterprises actually evaluate this, they can ask themselves a few questions: How much intelligence does this task require? How fast a response? How high is the cost of an incorrect result? For example, in scenarios such as healthcare and finance, accuracy and reliability are hard requirements.

Wang Yisu: A Token itself is more like a basic input; how much value it can ultimately generate depends on which tasks it is applied to and how enterprises integrate it into business processes. Based on my observations, AI scenarios can be roughly divided into two categories.

One is horizontal scenarios, that is, general needs that exist across industries, such as enterprise knowledge assistants, customer service Agents, office assistants, coding assistants, and so on. Such applications have broad coverage, and their value is more reflected in helping employees improve efficiency and reduce repetitive work.

The other is vertical industry scenarios, that is, applications that combine specific industry knowledge and business processes, such as investment research analysis in finance, assisted diagnosis in healthcare, and equipment operations and maintenance in manufacturing. Such scenarios usually require AI to understand more specialized data processes and business logic. In both types of scenarios, Tokens can directly generate economic value.

Q: What is Token economics? Why has it become an increasingly important framework for enterprises when discussing AI today?

Dr. Lai: From NVIDIA's perspective, Token economics refers to the economics behind Token generation and use. As data centers evolve from places that store and process data into AI factories dedicated to producing intelligence, Tokens are the product units produced by AI factories.

Token economics must answer not only "how much is a Token worth" but four things: first, how Tokens are valued, that is, how much the intelligence and interactive experience they carry are worth; second, how they are consumed, that is, who uses them, how many, and in what scenarios; third, how they are supplied, that is, how infrastructure can be used to produce Tokens efficiently and at low cost; and finally, how they are monetized, that is, how they are ultimately converted into revenue and business value.

This concept is becoming increasingly important largely because AI is moving from the pilot and large-scale training stage into the stage of scaled application and continuous inference. With the development of AI agents, a task may require longer context, more rounds of reasoning, and tool calls, and both the demand for Tokens and the way they are used will change greatly.

Once Tokens enter scaled production and use, enterprises need to pay attention to output, efficiency, and unit cost just as they would manage a production system. From this perspective, Token economics connects AI's technical inputs with economic outputs. Enterprises must consider both whether Tokens can be generated more efficiently and at lower cost, and where these Tokens are used and what kind of business applications and value creation they can ultimately support.

Wang Yisu: Let me start with a number—a number that makes me think Token economics must be seriously discussed. From early 2024 to March 2026, China's average daily Token calls soared from 100 billion directly to 140 trillion, a 1,400-fold increase in two years.

Behind this number, three things are happening at the same time.

First, enterprises used to habitually look at computing power investment; now they need to look at intelligent output. The old logic was how many cards you bought, how many kilowatt-hours of electricity you used, and how many data centers you built—this was accounting from the input dimension. After the emergence of Token economics, enterprises need to calculate another account: I spend 1 dollar to buy Tokens—how many tasks did they help me complete? How much time did they save? How much revenue did they create? This is accounting from the output dimension. From input to output, this is the turning point where the entire AI industry switches from a technology race to commercial implementation.

Second, new measurement units have emerged inside enterprises to re-manage AI investment. The most typical example is that this year a company proposed DAA, the number of daily active agents. Why does this thing exist? Because an enterprise only looking at how many Tokens it burned is the same as a steel plant only looking at how much coal it burned—how much coal is burned does not represent whether steel output is good. IDC also issued a DAA report, and we have also observed that some operators have begun to redesign their business models based on "number of users + AI terminal penetration rate + Token consumption + economic value per unit Token."

Third, AI management inside enterprises is becoming a joint matter for the CFO and CIO. In the past, AI investment was managed by the CIO alone, but now the scale of Token consumption has reached a level where the CFO must get involved. More and more enterprises are beginning to set up dedicated positions to manage AI resource consumption. This position requires understanding both models and finance, and the question to answer is: what exactly did each AI investment get back?

To summarize, Token economics is the inevitable product of enterprise AI investment transitioning from the computing power era to the intelligence era. Over the past year, this concept has gone from discussion among a few people to an industry consensus, which is why it has today become an increasingly core framework when enterprises discuss AI.

Question: When agents enter enterprises, what changes will happen to Token usage? What are the common categories of agent applications today, and how do their Token requirements differ?

Dr. Lai: From the perspective of workloads and actual use cases, I would roughly divide common agents into three categories.

First is the conversation and assistant type, such as customer service assistants and knowledge Q&A. These applications need timely responses, have relatively high latency requirements, need to support a large number of concurrent users, and each task is relatively short, so Token consumption is relatively easy to predict.

Second is the workflow-type agent, such as having an agent help you complete a research report. It needs to first understand the task, then retrieve materials, call tools, analyze information, and generate results. It generally goes through multiple rounds of model calls, and the context is also relatively long.

Third is the more autonomous, long-running agent, including multi-agent collaboration. They may run continuously for tens of minutes, several hours, or even longer, constantly calling various tools, updating context, and even launching sub-agents during the process.

We at NVIDIA have done some tests ourselves. An agent task may involve 50 to 500 or even more consecutive model calls. In multi-agent scenarios, Token consumption will be further amplified. So after agents develop, Token demand becomes increasingly difficult to predict—how long the context is, how many rounds of reasoning are needed, how many times tools are called, how many agents run simultaneously, and what requirements there are for response speed. All of these will change the structure of Token demand.

Wang Yisu: Let me add a trend judgment. In the next one to one and a half years, AI office agents are very likely to become the largest scenario of enterprise Token consumption, because office is one of the few scenarios that has both high-frequency usage and clear willingness to pay. In Q2 2026, the capital expenditure of some leading technology companies surged by 70% to 300%, among which AI office accounted for no small share. In Dr. Lai's classification, office agents are moving from workflow type to autonomous collaboration type, and Token consumption will be further amplified.

Question: To truly apply Token economics in business, do enterprises have a framework they can use for decision-making?

Dr. Lai: When enterprises consider AI investment, the core questions to answer are: Has this investment produced value? How many resources need to be invested? How can stable operation of AI capabilities be ensured? Can it ultimately be converted into business returns? Around these questions, we can understand them using the four pillars of Token economics proposed by NVIDIA.

First is the value of Tokens: whether AI output truly solves business problems. Second is the demand for Tokens: enterprises need to understand the demand for Tokens among different users and in different scenarios, including usage scale, task types, peak demand, and so on. Third is the supply of Tokens: enterprises need to focus on how to improve Token supply efficiency and reduce unit costs through model, computing power, and infrastructure planning. Finally is how Tokens are monetized, which many people care about: how to convert Token usage into actual business value and commercial returns. By considering these four pillars together, enterprises can evaluate Token investment relatively comprehensively from multiple dimensions of Token demand, supply, and value.

Wang Yisu: These four pillars are not just about technology; they are also core issues of enterprise operations, so they also need to be designed from the top-level architecture. I map them to four roles inside the enterprise, and I personally find this mapping quite useful.

The CEO and CFO care about monetization: how much revenue and efficiency each AI investment brings back, and the accounts must be calculated clearly. Business leaders care about value: whether AI output can actually be used by business teams. Cases where the demo is impressive but no one uses it in practice are all too common. Product and operations care about demand: whether peak loads can be withstood. Every additional ticket handled by a service-oriented agent consumes 5,000 to 15,000 more Tokens. For a company with 5,000 employees handling 10,000 tickets per day, that is 50 million to 150 million Tokens per day. The technical team cares about supply: whether delivery can be fast and stable.

The relationship among the four roles is "push up, pull down": the broader the business, the greater the demand; the greater the demand, the more the technology must deliver; and ultimately, technology investment must be judged by monetization. Many advanced enterprises in China already hold an AI business review every two weeks, bringing the four roles to the same table. This action is small, but it is the watershed that upgrades AI investment from a technical issue to a business issue.

Question: As an enterprise manager, how should one reasonably plan and manage the different Token demands across the whole company, so as to maximize Token usage efficiency while reducing costs?

Dr. Lai: This is indeed a relatively important and challenging question. When enterprises plan Token demand, they can no longer simply estimate it as "number of users × Tokens per request." In real production environments, AI workloads are very complex. Agents may require multiple rounds of reasoning, and different task complexities also affect Token consumption. Token demand forecasting can be analyzed at three levels.

The first level is baseline demand estimation: establish an initial demand baseline through the number of users, number of requests, and the approximate Token scale per request.

The second level is workload multiplier factors: further consider factors such as the number of reasoning rounds, number of agent calls, retries, and caching.

The third level is operational demand: combine peak demand, business growth, and latency requirements in actual deployment to help enterprises plan infrastructure more accurately. Through these methods, enterprises can gradually move from simple estimates to Token demand that is closer to actual scenarios, and plan AI infrastructure investment more reasonably.

Wang Yisu: Unclear demand is an extremely common and fatal problem. Over the past two years, the thing I have heard most is "I want to adopt AI," but no one has a sense of what exactly it is used for or how good the results are. This year is already much better. Companies that do well let people on the business lines truly build AI products themselves.

I just saw a case this year: the finance department set up a dedicated AI-oriented IT department. They built their own agent system, connected procurement, approval, and profit margin calculation into one table, and also brought in external professional suppliers. But do not over-trust the capabilities of non-IT personnel. Many business people use AI Coding to build things themselves, and the maturity varies enormously. Professional people should still do professional work, and there must be overall top-level planning.

When it comes to budgeting, Dr. Lai also mentioned some points just now. My suggestion is three sets of budgets. The first is the baseline budget: based on the current scale and model forecast, provide the average monthly Token consumption for the next 12 months. The second is the growth budget: reserve a growth margin each month, whether 10%–20% or 30%–40%, and observe it dynamically, because AI usage demand is constantly being explored and increased. The third is the peak budget: when new products launch, seasonal peaks occur, or batch tasks concentrate, there will be instantaneous consumption. Backup must be done well here so that user experience does not collapse at critical moments.

Only when these three budgets are added together is that the Token capacity an enterprise should truly prepare. Why is this important? Because I have seen too many enterprises "just get it running first and talk later," and then one day suddenly face an astronomical bill of several million Tokens. So-called advance planning is not a waste of time; it leaves room for trial and error.

Question: After demand is provided, enterprises still need to judge whether this infrastructure investment has produced results. How can one judge whether Token investment is effective? What indicators are worth referencing?

Wang Yisu: I divide it into three circles.

The outermost layer is business outcome indicators, looking at five: task completion rate, time saved, output quality, usage rate, and business return. Looking only at Token consumption is absolutely not enough.

The middle layer is process efficiency indicators, used to fill in the attribution gap: automation rate, cycle compression rate, manual intervention rate, and error rework rate. A supermarket chain giant used AI to negotiate supplier contracts, saving 3% in costs, achieving 4x ROI, and extending payment terms by 35 days. This was calculated by mapping each indicator to each stage.

The bottom layer is infrastructure indicators, which determine the ability to deliver continuously and stably.

How do the three circles work together? My summary is: calculate costs from bottom to top, then look at value from top to bottom. The bottom-layer infrastructure tells the enterprise "how many Tokens are needed to have this capability"; the middle-layer process indicators tell the enterprise "how many processes were run using these capabilities"; and the top-layer business indicators tell the enterprise "how many results these processes brought back."

Dr. Lai: I will add a little on efficiency from the third circle. As models become larger and the tasks undertaken by agents become more complex, how to process Tokens faster, more stably, and at lower cost becomes key to measuring the effectiveness of investment. Behind this inevitably lies the collaborative efficiency of AI infrastructure, including both hardware and software.

At the hardware level, people tend to focus on compute power. Today I will not talk about compute power here; instead, I want to emphasize the importance of networking. When a model needs to work collaboratively across multiple GPUs and multiple nodes, a high-bandwidth, low-latency network can reduce waiting time and enable different compute units to cooperate more efficiently. This efficiency improvement will ultimately be reflected in higher Token processing speed, lower response time, and lower cost per Token.

At the software level, inference optimization also greatly affects the economics of Tokens. With the same compute investment, more efficient inference capability means the system can process more Tokens and achieve higher overall resource utilization.

Specifically, first, efficient inference frameworks are needed, such as the open-source Hugging Face, vLLM, or NVIDIA's TensorRT-LLM. They apply various optimizations to the model inference process, including operator performance optimization, quantization, efficient memory management, and so on. Efficient serving frameworks are also needed, such as NVIDIA's Dynamo. It targets AI agents and large-scale inference scenarios, decoupling Prefill and Decode onto different GPUs through disaggregated serving, and further optimizing scheduling and resource allocation across different workloads with technical means such as KV-aware routing and KV Cache offloading, improving system throughput and overall resource utilization.

Finally, enterprises can conduct a comprehensive evaluation combining business outcomes and technical metrics, such as cost per million Tokens, response speed, system throughput efficiency, and task completion quality, to judge whether AI infrastructure investment truly creates sufficient value.

Q: Finally, please look ahead to the future. Token economics is developing rapidly as a topic. Where do the two of you think it currently stands? What key issues remain to be solved in the future before it can truly unlock the productivity value of the AI era?

Dr. Lai: To see what stage Token economics has reached, one can look at the changes in AI applications themselves. At the earliest stage, everyone focused on compute power and inference cost, such as how many GPUs are needed, how many Tokens can be generated per second, and the cost per million Tokens. The core problem solved at this stage was: how to produce Tokens more efficiently and at lower cost.

We are now entering the second stage: AI agents and enterprise-level applications are developing rapidly, and Token demand is beginning to diversify. Different tasks have different requirements for context length, number of inference rounds, response speed, concurrency, and throughput. What enterprises need to focus on is no longer just the unit cost of Tokens, but also how many Tokens are needed, when they are needed, which models and infrastructure should provide them, and how to find a balance between performance and cost.

Looking further ahead, Token economics will increasingly move toward value orientation. Once Tokens can be produced at scale, enterprises will naturally ask further: what tasks did these Tokens ultimately complete? How much efficiency was improved? How much business value was created? So we are now in a very critical transitional stage, where enterprises are gradually moving from focusing on the production cost and efficiency of Tokens themselves to simultaneously managing the demand, supply, and value of Tokens.

For infrastructure, there are still many problems to solve next: inference efficiency, energy efficiency, resource scheduling, large-scale operation, and so on. For the entire industry, the bigger challenge is how to further transform improvements in technical efficiency into business value that can be continuously measured and managed.

First, Token measurement standards need to be unified. Now in China alone there are already more than 2,000 APIs for specialized agents, and each company has somewhat different definitions of Tokens, measurement granularity, and calculation rules. The good news is that some companies are already trying to achieve underlying standardization through AI + eSIM, uniformly recording Token consumption at the chip level, but this is still at an early stage.

Second, the AI operating organization within enterprises has not yet been established. As just mentioned, some advanced enterprises have begun setting up dedicated positions to manage AI resource consumption, but most enterprises still remain at the understanding that "AI is a matter for the technical department." Real AI transformation is a transformation of operations, not just a technology upgrade. I think this is the most core difference in this wave of large AI models.

Third, collaboration protocols between agents have not yet been established. In the future, Token consumption will not only occur between people and AI, but more between agents and agents. In the future, there may be a situation where an enterprise has 1,000 agents calling each other and trading Tokens with each other. But the current A2A (Agent to Agent) protocol does not yet have industry consensus.

Regarding Token economics, the key proposition for the entire industry is: can we establish an infrastructure for value-based pricing, Token measurement standards, enterprise AI organizational operations, and agent-to-agent protocols? If these things are achieved, the AI industry will truly enter the era of productivity.

Source

TMTPost · AGI