Inference computing power demand is rising rapidly: AI computing servers are in short supply, and a new trillion-level blue ocean market has emerged
A report by the China Academy of Information and Communications Technology (CAICT) predicts that by 2026, inference computing power will account for 70.5% of the total. China's average daily Token consumption has surged from 100 billion in 2024 to 500 trillion in August this year, and the focus of computing power demand is shifting from training to inference. AI computing servers are already noticeably in short supply. Industry players advocate improving Token cost-effectiveness through software optimization; for example, DeepSeek's dual-path KV Cache loading mechanism increases the operating efficiency of online agents by 1.96 times. In 2026, a group of ASIC inference chip startup teams has emerged, and at the same power consumption and cost, inference performance has the opportunity to reach 10 to 20 times that of GPUs.
Body
The English body is being prepared. You can open the original source.