DeepSeek open-sources inference framework DeepInfer and Token codec component deepseek-recipe

DeepSeek open-sources its self-developed inference framework DeepInfer (Apache 2.0) and the production-grade Token codec component deepseek-recipe (MIT), supporting DeepSeek V4.1 Flash in serving 155 trillion Tokens per day on a single cluster. Official online statistics show an average of 77K Tokens input per request, a time to first token of about 400 milliseconds, 230 Tokens output per second during the generation phase, and an average of 96% of input hitting the cache. DeepInfer currently supports V4.1 Flash; the open-source version provides only offline inference, and the KV cache management and load balancing code has not yet been fully open-sourced because it is tied to internal infrastructure.