Understand this story
On October 20, 2026, DeepSeek open-sourced its internal inference framework DeepInfer. According to Yuchen Jin, citing data disclosed by DeepSeek, a single cluster processes 155 trillion tokens per day and generates $38 in revenue per GPU-hour (off-peak pricing); DeepInfer supports both NVIDIA Blackwell and Huawei Ascend. A subsequent 36Kr report stated that its accompanying open-source deepseek-recipe supports NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. On October 21, swyx, citing Latent.Space data, said a single cluster processes 155T tokens per day with a 96% input cache hit rate, and argued that the lifetime value of each sold GPU has risen by about 30%. On October 24, Nathan Lambert said DeepSeek's single cluster processes 150T tokens per day, exceeding the combined open-source model inference volume of OpenRouter, Fireworks, Together, and BaseTen, and noted that the actual number of clusters is unknown; on the same day, LMSYS published a same-hardware benchmark comparison, stating that SGLang meets or exceeds DeepInfer on DeepInfer's own benchmark, with prefill throughput up to 1.04x and decode step length shortened by 1.1%–7.4% on 64x GB300.
AI-generated from public reports
Latest development · Report
After DeepSeek Open-Sourced DeepInfer, vLLM and SGLang Update Inference PerformanceDevelopments
4 confirmed developments
Nathan Lambert says DeepSeek processes 150T tokens per day on a single cluster
Source:Nathan LambertLMSYS publishes same-hardware benchmark comparison of SGLang and DeepInfer
Source:LMSYS Blogswyx says the lifetime value of every GPU sold rose about 30% today
Source:Swyx- Source:Yuchen Jin
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 26
36Kr · AI
After DeepSeek Open-Sourced DeepInfer, vLLM and SGLang Update Inference Performance
Less than a week after DeepSeek open-sourced its inference engine DeepInfer, vLLM and SGLang successively released optimized versions, with performance approaching or even partially exceeding DeepInfer under their respective test configurations. Under a decoding configuration with a 131K context and 3 draft tokens per round, vLLM measured on GB200 showed that at batch sizes from 8 to 256, the throughput gap with DeepInfer was only 6%, while prefill throughput for newly added tokens improved by 35.6%–43.9%; on a GB300 NVL72 rack, the SemiAnalysis AgentX test achieved per-GPU throughput of 954,000 tokens/s, 7.9 times the previous single-node result. SGLang tested prefill on 4 B300s and 4 GB300s, with peak throughput about 4% higher than DeepInfer; in a 64-GB300 decoding test with 256 requests per GPU, the single-step latency was 28.88 milliseconds, about 7.4% shorter than DeepInfer's 31.19 milliseconds.
Sina Tech
DeepSeek V4.1-Flash-1017 Tops OpenRouter Weekly Call Volume, Up 801% Week-over-Week
OpenRouter's weekly rankings for October 19–25 show that DeepSeek's V4.1-Flash-1017, updated on October 17, topped the list with 33.6 trillion tokens in weekly call volume, up 801% week-over-week. This version continues the asymmetric MoE architecture with 552B total parameters, 8B input activation, and 16B output activation, with a focus on optimizing code, automated agents, and vision tasks; the previous V4.1 Flash recorded 18 trillion tokens in weekly call volume, down 45% week-over-week. Last week's global total call volume was 188 trillion tokens, up 14.63% week-over-week, and domestic Chinese models, at 93.54 trillion tokens, remained above US models for the twenty-sixth consecutive week.
Leiphone
DeepSeek Open-Sources Inference Framework DeepInfer; AI-Written ForgeKernel Tops Operator Benchmark in the Same Week
DeepSeek recently fully open-sourced its inference framework DeepInfer, further opening up the model execution pipeline after the model weights and operator libraries such as FlashMLA and DeepGEMM. As of October 22, ForgeKernel—entirely written by AI with no manual line-by-line tuning—simultaneously topped the NVIDIA SOL-ExecBench FlashInfer-Bench combined leaderboard and the Ascend CANNBench A3 hardware leaderboard, with a benchmark speedup of 1.835×, 25% ahead of the runner-up. On the operators for DeepSeek v4.1 Flash, ForgeKernel, built from scratch through AI iteration, outperformed the NVIDIA version open-sourced by DeepSeek.
36Kr · AI
In the Global Open Source War, How Many AI Players Will Be Exposed?
Using DeepSeek's open-sourcing of its internal inference framework DeepInfer as an entry point, the article discusses the phenomenon of large model vendors worldwide collectively pivoting to open source. It states that DeepInfer is released alongside the open-source deepseek-recipe, supports NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, and cites technical documentation disclosing roughly 155 trillion Token requests per day for a single cluster, with peak throughput approaching 1.794 billion Tokens per second. Drawing on the precedents of Microsoft's embrace of open source and Android's tiered openness, the author analyzes open source as a competitive lever for expanding ecosystems, while pointing out that closing the commercialization loop, compliance, and community governance are real challenges facing open source players.
Oct 25
Recode China AIFirst-party
China AI Weekly: DeepSeek's roughly $15 billion funding, Kimi K3.1 Pro release, and employment policy adjustments
The China AI Weekly Digest summarizes 306 China AI news items tracked from October 16 to 23, 2026. According to Reuters, DeepSeek's latest funding round raised about 100 billion yuan, with a pre-money valuation of 500 billion yuan; Tencent and CATL were the main investors. Moonshot AI released Kimi K3.1 Pro with a million-token context window and said the Highspeed version is two to three times faster. The General Office of the State Council issued AI employment policy guidelines, proposing 18 tasks covering new profession cultivation, AI training, and employment risk monitoring.
Yicai
AI Weekly: Meta's Muse surpasses 10 million downloads; CXMT and Micron diverge on technology
Meta's AI assistant Muse has accumulated 10.9 million downloads since its September launch, of which 10.2 million came from the United States, making it one of the mobile apps to surpass 10 million U.S. downloads within 40 days of launch. DeepSeek open-sourced its inference framework DeepInfer and disclosed that a single cluster serves a throughput of 155 trillion tokens per day, with revenue of $38 per card per hour. Anthropic launched Claude Duet, a personal agent based on Claude Opus 5.5, opening research preview waitlist applications to Claude Pro and Max subscribers in the United States.
TMTPost · AGI
CATL earned a net profit of 21.2 billion yuan in a single quarter, yet fell nearly 15% in two days: what exactly is the AI rally killing?
CATL's third-quarter net profit attributable to shareholders was 21.246 billion yuan, up 14.54% year on year, below the lower bound of institutional forecasts. On October 21 its A-shares closed down 12.03%, and over two trading days they fell nearly 15% cumulatively, wiping about 140 billion yuan off its market value. This round of adjustment is weighing on valuations rather than already-realized earnings: the ChiNext Index's PE-TTM fell from about 54 times in June to about 32 times, while the STAR 50 dropped from about 264 times to above 110 times. Over the same period, equity ETFs saw net inflows for eight consecutive trading days, totaling 94.753 billion yuan as of October 14.
Oct 24
Wallstreetcn
DeepSeek did the math: $38 per GPU-hour — can the inference business make money?
When DeepSeek open-sourced its inference framework DeepInfer on October 20, it announced that a single cluster can process 155 trillion tokens per day, which, converted at DeepSeek API off-peak pricing, corresponds to an average of $38 in revenue per GPU-hour. The article notes that this is theoretical billed revenue; DeepSeek has not disclosed the cluster's hardware composition, cost bills, or the share of paid traffic, and DeepInfer currently mainly targets DeepSeek-V4.1 Flash. The article further compares the R&D and inference expenditure structures of OpenAI and Anthropic, as well as cases such as CoreWeave, Cursor, and Character.AI, to show that high output value per GPU does not mean inference services are necessarily profitable.
LMSYS BlogFirst-party
Beyond DeepInfer: What SGLang Delivered for DeepSeek-V4.1-Flash Inference
SGLang matches or exceeds DeepInfer on DeepInfer's own benchmarks: up to 1.04x prefill throughput on 4x GB300 and 4x B300, and 1.1%–7.4% shorter decode step time on 64x GB300. SGLang has integrated DeepSeek's Mega series kernels, DSpark speculative decoding, SWA bounded replay, and two Engram placement methods, and has filled in KV cache, prefill/decode disaggregation, and inter-replica load balancing that DeepInfer does not include. SGLang supports NVIDIA GB300, B300, B200, H200, and AMD MI355X.
Nathan Lambert
Nathan Lambert: DeepSeek processes 150T tokens per day on a single cluster, more than the combined total of several open-source inference platforms
Nathan Lambert says DeepSeek processes 150T tokens per day on a single cluster, more than the combined open-source model inference volume on OpenRouter, Fireworks, Together, and BaseTen. He also posted a reconstructed comparison chart of his own based on these companies' inference volume reports, and noted that this figure corresponds to only a single cluster; it remains unclear how many clusters DeepSeek actually has.
Oct 21
Swyx
swyx on DeepSeek GPU revenue data: lifetime value per card up about 30%
Citing DeepSeek production data disclosed by Latent.Space, swyx says a single cluster processes 155T tokens per day with a 96% input cache hit rate, yielding $38 in revenue per GPU-hour at off-peak prices. He believes this raises the lifetime value of every GPU sold by about 30%, corresponding to roughly $600 billion in market value potential, and jokes that the more you buy, the more you earn.
Yuchen JinSelected
DeepSeek's single cluster processes 155 trillion tokens per day, generating $38 in revenue per GPU-hour
Yuchen Jin, citing data disclosed by DeepSeek, said that a single cluster currently processes 155 trillion tokens per day and generates $38 in revenue per GPU-hour (off-peak pricing). He noted that DeepSeek has not only open-sourced its inference engine but also disclosed revenue figures, though it did not specify which GPU models were used; DeepInfer supports both NVIDIA Blackwell and Huawei Ascend. Based on an estimated B300 rental price of $7 per GPU-hour, the gross margin is approximately 82%.
Heat trend for this story
- Current
- 5
- Period peak
- 15 Oct 26 23:00
- 24-hour change
- —
Move the pointer or select the chart to inspect hourly heat. Use the left and right arrow keys with a keyboard.