DeepSeek Fully Open-Sources Its Internal Inference Framework DeepInfer
English text is being prepared.
DeepSeek announced that it is fully open-sourcing its internal inference framework DeepInfer, and is simultaneously open-sourcing the production-grade token codec component deepseek-recipe, whose processing logic is fully aligned with the official API. DeepInfer integrates high-performance operators and an inference engine, supports capabilities such as distributed inference, speculative decoding, SWA Bounded Replay, and structured output, and is compatible with NVIDIA and Huawei Ascend platforms. DeepSeek stated that it has already used this framework to deliver high-throughput, low-latency large-scale inference services for DeepSeek V4.1 Flash, and that inference code for future new models and subsequent optimizations will continue to be updated.
Why read this
AI score 80
DeepSeek has open-sourced its internal inference framework and provided codec components aligned with the official API, offering insight into its large-scale inference deployment experience.
Topics
Body
10 月 20 日消息，DeepSeek 今日官宣，在此前开源一系列大模型、TileLang 编译工具及推理算子库的基础上，决定进一步将内部推理框架 DeepInfer 全面开源。
DeepSeek 官方表示，大模型的能力边界由模型本身划定，但如何以合理成本提供超大规模的模型服务，还取决于一个长期未被公众了解的中间层：推理框架。
DeepSeek 希望通过分享内部的部署经验，帮助开发者与企业组织优化大规模推理的吞吐与延迟，从而高效构建私有化及公有云部署，降低算力成本、减少能源消耗，让智能以更低的价格触达更广泛的人群，实现真正的大模型普惠。
基于 DeepInfer，DeepSeek 官方已实现 DeepSeek V4.1 Flash 高吞吐、低延迟的大规模推理服务。未来，DeepSeek 新模型的推理代码及后续优化也将持续更新。
DeepInfer 推理框架：集成高性能算子与深度优化的推理引擎，支持分布式推理、推测解码、SWA Bounded Replay、结构化输出等能力，兼容英伟达与华为昇腾平台。
deepseek-recipe 编解码组件：同步开源的生产级 Token 编解码组件，与 DeepSeek 官方 API 处理逻辑完整对齐，确保社区获得与官方部署一致的 API 体验。
附开源地址如下：