Understand this story
On October 29, 2026, DeepSeek open-sourced the inference engine DeepInfer on GitHub; this engine is the inference engine behind DeepSeek-V4.1-Flash. Leiphone reported the same day that DeepInfer includes technologies such as MoE expert parallelism, asynchronous batch scheduling, custom kernels, and speculative decoding, and disclosed that a single cluster needs to process approximately 155 trillion tokens per day. Its Zero-Overhead Batch Scheduler lets the CPU schedule computation one round in advance, overlapping with GPU forward passes, and handles cross-round state misalignment through total_inflight_tokens accounting, Remap mapping, and DeferredRelease deferred reclamation. Currently, only offline inference capabilities are open; components such as online KV Cache storage and cross-replica load balancing have not yet been fully disclosed, and the official team has not separately published end-to-end speedup data for this scheduling mechanism relative to synchronous solutions.
AI-generated from public reports
Developments
1 confirmed developments
- Source:English text is being prepared.
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 30
36Kr · AI
MaaS Has Arrived: Anthropic Turns an Operating Profit for the Quarter, China's Public Cloud MaaS Revenue Reaches 10.63 Billion Yuan in Half a Year
Anthropic's revenue in the second quarter of this year was 11.555 billion US dollars, achieving an operating profit of 974 million US dollars for the first time, with about 80% of revenue coming from pay-as-you-go billing. IDC data shows that in the first half of 2026, China's public cloud MaaS market revenue reached 10.63 billion yuan, up 889.4% year on year, and its share of public cloud PaaS revenue rose from 5.2% in the second half of last year to 21.0%. Zhipu's open platform and API revenue was 825 million yuan, accounting for 86.5% of total revenue; Volcano Engine ranked first with a 45.1% share. The article argues that model services can now generate gross profit, and the unit of competition has shifted to inference optimization and delivery capability.
Oct 29
Leiphone
DeepSeek Open-Sources Inference Engine DeepInfer, Discloses That a Single Cluster Processes About 155 Trillion Tokens per Day
DeepSeek has open-sourced its inference engine DeepInfer on GitHub—the inference engine behind DeepSeek-V4.1-Flash—which includes technologies such as MoE expert parallelism, asynchronous batch scheduling, custom kernels, and speculative decoding, and discloses that a single cluster needs to process about 155 trillion tokens per day. Its Zero-Overhead Batch Scheduler lets the CPU schedule computation one round ahead, overlapping with GPU forward passes, and handles cross-round state misalignment through total_inflight_tokens accounting, Remap mapping, and DeferredRelease deferred reclamation. What is currently available is offline inference capability; components such as online KV Cache storage and cross-replica load balancing have not yet been fully disclosed, and the official team has not separately published end-to-end speedup data for this scheduling mechanism relative to synchronous solutions.