CAICT releases report on performance monitoring of large model token services: domestic daily average consumption in June was about 500 trillion

CAICT: Domestic daily token consumption in June was about 500 trillion, nearly 4 times that of December 2025

2026-10-24 22:41 IT Home - Haomiao

Thanks to IT Home user "A Different Experience" for the tip-off!

IT Home, October 24 news: According to the China Academy of Information and Communications Technology (CAICT), with the rapid iteration of large model technology, token services have become a key point in the transition from technology to application, and China's daily average token call volume has continued to climb. In June 2026, domestic daily average token consumption was about 500 trillion, nearly 4 times that of December 2025. More and more MaaS (Model as a Service) platform vendors in the industry are supplying large model token services through public cloud APIs. However, users lack intuitive criteria for judging token service performance and face various problems such as difficulty in selecting services.

To effectively measure service performance, the Institute of Artificial Intelligence of the China Academy of Information and Communications Technology (hereinafter referred to as "CAICT"), the Key Laboratory of Artificial Intelligence Large Models and Software/Hardware Evaluation of the Ministry of Industry and Information Technology, and the China Artificial Intelligence Industry Development Alliance have, since 2025, continuously carried out performance monitoring of public cloud large model token services to provide an important reference for industrial applications. They now jointly release the "Large Model Token Service Performance Monitoring Report (2026)."

The report is based on monitoring data from January to August 2026 and analyzes dimensions including the call success rate, throughput speed, time to first token, stability, price, and context window of token services on various platforms. It compares the performance differences between the official platforms of original large model developers and the four open-source models DeepSeek, Qwen, GLM, and Kimi on third-party platforms, further analyzes the key factors affecting token service performance, summarizes core industry observations, and clarifies the next work plan, aiming to provide a reference for users in selecting services and to offer direction for the industry to optimize service capabilities.

IT Home attaches the core viewpoints of the report:

1. Domestic token service throughput per second has doubled compared with 2025, but remains lower than abroad

The report states that in August, the average TPS (tokens output per second) of domestic models reached 78.23 (tokens/second), doubling compared with the 2025 average of 38.25 (tokens/second). However, in August, the average TPS of typical foreign models was 112.82 (tokens/second), also more than double the 51.35 (tokens/second) in December 2025, and overall better than domestic models.

▲ Monthly average TPS of all large model token services

2. Token service latency has increased, with consistent trends at home and abroad

The report states that since the beginning of this year, the average TTFT (time to first token) of domestic models has shown a continuous upward trend, reaching an average of 1.26 seconds in August, a marked increase compared with the 2025 average of 0.78 seconds. But at the same time, the average TTFT of typical foreign models reached 2.08 seconds in August, also a marked increase compared with 1.04 seconds in April. It is worth noting that the rise in TTFT is not simply a performance degradation, but rather the result of multiple factors occurring simultaneously, including increases in model parameter count, context length, and call volume.

▲ Monthly average TTFT of all large model token services

3. Token service prices rebounded markedly in 2026

The report states that after the price wars of the past two years, model service prices at home and abroad rose markedly in 2026. The normalized average price in China in the second half of 2025 was 6.8 yuan, while the average price so far this year is 13.46 yuan, an increase of nearly 1 times. However, foreign prices remain higher than domestic ones; the average normalized price of typical models released since 2026 is about 117.8 yuan.

▲ Trend chart of normalized prices at release of domestic original large models

4. Model context lengths continue to expand toward the million-token level

The report states that among the monitored models, those with a context length of 1M (1 million tokens) account for 47.8%, and those with 200K and above total 95.7%, an increase of more than 76 percentage points compared with the end of 2025, while the proportion of models below 200K continues to decrease.

▲ Trend chart of normalized prices at release of domestic original large models

5. Open-source model token service performance improved significantly, and the same model fluctuates greatly in performance across different platforms

The report states that in May, new versions of four open-source models were updated for monitoring, and the results showed that TPS continued to improve and TTFT latency continued to shrink. For example, in August, the TPS of DeepSeek-V4-Pro had reached twice that of DeepSeek-V3.2. In terms of latency, after the main models completed generational upgrades in May, TTFT generally rose temporarily, and as inference optimization measures were implemented, latency indicators gradually fell back to pre-upgrade levels. At the same time, the four models showed differing TPS, TTFT, and call success rates across different platforms. Some third-party platforms performed significantly better than the model services provided by official platforms, and the TPS gap for the same model across different platforms could be nearly double at its largest. Differences among platforms in inference acceleration, optimization strategies, and cost control for the same model are the main reasons for the performance divergence.