DeepSeek open-sources inference framework DeepInfer and codec components under Apache 2.0

A couple of days ago, the whole internet was still guessing how DeepSeek V4.1 Flash managed to push each million input tokens down to $0.15 and output to $0.6, and today they lifted the veil themselves: DeepSeek open-sourced, under the Apache 2.0 license, the inference framework DeepInfer that powers its own online services, along with the API-identical codec component. What they released this time is the core foundation they use to run large models online themselves, and it directly packs in high-performance operators, distributed inference, speculative decoding, SWA Bounded Replay, and structured output; the company even promises that inference code for future new models will continue to be merged into this repository. In the past, when enterprises deployed open-source models privately on-premises, the biggest headache was that the official side released the weights, but the service you built yourself with general-purpose frameworks not only had slow throughput and exploding VRAM, but, most critically, the format and tool-calling behavior produced by the same prompt always failed to match the official online API. And the treasure in this open-source release that is easiest for outsiders to overlook but most valuable for engineering deployment is actually the deepseek-recipe that came along as a bonus: it hands over the Token encoding and decoding logic that the official online API is using, unchanged, directly smoothing out the behavioral temperature difference between local environments and the official interface for all teams doing private deployment. It's like a Michelin restaurant: in the past, open-sourcing model weights was just printing the recipe for you for free, and when you cooked it at home it always lacked that wok hei. Now DeepSeek has packaged up its own kitchen's custom high-pressure high-heat stove, DeepInfer, together with the chef's standardized prep and plating process, deepseek-recipe, and given them all to you. However, I went and dug through the GitHub repository's README myself, and found that even amid the whole internet cheering for completely open source, one should stay clear-headed; the repository documentation is very candid. What was released this time is actually only the offline inference engine. The three most core pieces of infrastructure that truly support a complete online high-concurrency service—the KV cache system, prefill-decode disaggregated deployment, and multi-replica load balancing—were not released this time; moreover, the hardware threshold is extremely high, as it currently recognizes only two top-tier compute cards, NVIDIA's latest Blackwell architecture SM100 and Huawei Ascend 950, and the only supported model is the just-released V4.1-Flash. Consumer-grade GPUs in ordinary users' hands simply cannot run it. But it is precisely this line in the README limiting hardware support to NVIDIA SM100 and Huawei Ascend 950 that exposes an industrial signal harsher than the framework itself: leading Chinese large-model companies are no longer waiting for chip vendors to slowly adapt to them; instead, they are stepping in themselves to write low-level operators, mastering Silicon Valley's top-tier Blackwell on one hand while welding domestic Ascend 950 directly into first-tier production standards on the other. To put it plainly, this means completely different actions for different people. If you are an architect doing enterprise-level private deployment or domestic compute adaptation, hurry up and thoroughly understand deepseek-recipe's codec alignment logic and low-level operators; that is where real money in saved tuning hours is hidden. As for individual developers with a single card in hand, there is no need to join the deployment excitement this time; honestly, calling their API for a few dimes is far more cost-effective than building your own cluster. When a company dares to give away even the blueprints of its own money-printing machine, what it is after is no longer the small margin it earns from you, but making the entire industry's production lines grow into its shape. I've broken down in the first post all the hardware thresholds and missing components that the README does not explicitly mention in the tweet; brothers preparing to use it for private deployment should take a look and compare first.

View original

前两天全网还在猜 DeepSeek V4.1 Flash 是怎么把每百万输入压到 0.15 美元、输出压到 0.6 美元的，今天他们自己就把谜底掀了，DeepSeek用 Apache 2.0 协议把支撑自家线上服务的推理框架 DeepInfer 和 API 同款编解码组件全给开源了出来。 这次放出来是他们自己线上跑大模型用的核心底座，里面直接塞进了高性能算子、分布式推理、投机解码、SWA Bounded Replay 和结构化输出，甚至连官方承诺以后新模型的推理代码也会持续往这个仓库里合入。 以前企业在本地私有化部署开源模型，最头疼的痛点就是官方放了权重，但你自己用通用框架搭出来的服务，不仅吞吐慢、显存炸，最要命的是同一个提示词跑出来的格式和工具调用行为，总是跟官方线上 API 对不上号。 而这波开源里最容易被外行漏看、但对工程落地最值钱的宝贝，其实是那个顺手附赠的 deepseek-recipe，它把官方线上 API 正在用的 Token 编解码逻辑原封不动交了出来，直接替所有做私有化部署的团队抹平了本地环境和官方接口之间的行为温差。 这就像一家米其林餐厅，以前开源模型权重只是把菜谱免费印给你，你自己在家炒总是差一口锅气， 现在 DeepSeek 连自家后厨定制的高压猛火灶台 DeepInfer，外加主厨切配摆盘的标准化工序 deepseek-recipe 一块打包送你了。 不过我顺手去扒了一遍 GitHub 仓库的 README，发现对着全网欢呼的完全开源也得留个清醒，仓库文档写得很实在， 这次放出来的其实只是离线推理引擎，真正撑起完整线上高并发服务最核心的三块基建：KV 缓存系统、预填充与解码分离部署、多副本负载均衡这次并没有放出来；而且硬件门槛极高，目前只认英伟达最新的 Blackwell 架构 SM100 和华为昇腾 950 两款顶级算力卡，支持的模型也只限刚发的 V4.1-Flash，普通玩家手里的消费级显卡压根跑不起来。 但恰恰是 README 里这行只支持英伟达 SM100 和华为昇腾 950 的硬件限定，暴露了比框架本身更狠的产业信号，头部国产大模型已经不再等芯片厂商来缓慢适配自己，而是亲自下场写底层算子，一边吃透硅谷最顶级的 Blackwell，一边把国产昇腾 950 直接焊进第一梯队的生产标准里。 说白了，这对不同的人意味着完全不同的动作，如果你是做企业级私有化部署或国产算力适配的架构师，赶紧去把 deepseek-recipe 的编解码对齐逻辑和底层算子吃透，那里藏着省下大把调优工时的真金白银， 至于手握单卡的个人开发者，这趟就不必跟着去凑部署的热闹了，老老实实调它家几毛钱的 API 比自己搭集群划算得多。 当一家公司连自己压箱底的印钞机图纸都敢往外送时，它图谋的早就不是赚你那点差价，而是让整个行业的生产线都长成它的形状。 我把 README 里没在推文上明说的硬件门槛和缺失组件全拆在一楼了，准备拿它做私有化部署的兄弟先对照看一眼。