DeepSeek 开源内部推理框架 DeepInfer
DeepSeek is open-sourcing DeepInfer, its internal framework for serving models at scale.
DeepSeek 开源了内部推理框架 DeepInfer，称其支撑 DeepSeek-V4.1-Flash 的高吞吐、低延迟推理。该框架同时支持 Nvidia 和华为昇腾硬件，具备分布式推理、投机解码、SWA Bounded Replay 和结构化输出等能力。DeepSeek 还一并发布了 deepseek-recipe，一个与官方 API 完全对齐的生产级 tokenization 组件。
查看原帖🚀 隆重推出 DeepInfer，我们的自研推理框架，现已全面开源！ 它以高吞吐量和低延迟大规模支撑 DeepSeek-V4.1-Flash。我们分享生产环境技术栈，以帮助社区降低计算成本、减少能源消耗，让 AI 更加普惠。 🔗 https://t.co/JGAik1kgk0 1/4
查看原文
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source! Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable. 🔗 https://t.co/JGAik1kgk0 1/4
正文
DeepSeek is open-sourcing DeepInfer, its internal framework for serving models at scale.
Inference frameworks coordinate compute clusters to shape AI throughput, latency, cost, and energy efficiency. DeepSeek says DeepInfer powers high-throughput, low-latency inference for V4.1 Flash.
Supporting both Nvidia and Huawei Ascend hardware, DeepInfer enables distributed inference, speculative decoding, SWA Bounded Replay, and structured outputs. Alongside the framework, DeepSeek also released deepseek-recipe, a production-grade tokenization component fully aligned with its official API.