AI News Daily

DeepSeek open-sources inference framework DeepInfer

DeepSeek has open-sourced DeepInfer, the high-performance inference framework supporting its online services, under the Apache 2.0 license. This release focuses on offline inference, is built primarily on vLLM, and draws on SGLang and TensorRT-LLM. It integrates previously separately open-sourced kernels such as FlashMLA, DeepGEMM, DeepEP, and EPLB into a complete engine, filling in system layers such as scheduling, parallelism, and speculative decoding, while supporting NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. The original article also breaks down designs including DSpark speculative decoding, YoCo SWA bounded replay, asynchronous scheduling, batch-invariant deterministic inference, and regex/JSON schema/EBNF structured output.

• English text is being prepared.Tianyi Cui
• English text is being prepared.The Paper