←Search results

Search “Reasoning”

By relevance · Total 680 results Clear search

Wednesday

Tuesday

  2. Tianyi Cui

    DeepSeek open-sources the inference framework DeepInfer and components such as deepseek-recipe

    DeepSeek has open-sourced its in-house inference framework DeepInfer, and simultaneously released key components for deploying DeepSeek models, including deepseek-recipe. DeepInfer already supports large-scale serving of DeepSeek-V4.1-Flash in production, with a focus on high throughput and low latency. DeepSeek said that publishing its production stack is intended to help the community lower compute costs, reduce energy consumption, and enable more organizations to deploy DeepSeek models.

    Matched fieldsTitle

    #hot-signal#English text is being prepared.#English text is being prepared.#DeepSeek
  3. Yuchen Jin

    DeepSeek Open-Sources Internal Inference Framework DeepInfer

    DeepSeek announced the open-sourcing of its internal inference framework DeepInfer, which supports large-scale operation of DeepSeek-V4.1-Flash and focuses on high throughput and low latency. DeepSeek said it is making this production stack public to help the community lower compute costs, reduce energy consumption, and make AI more affordable, and included a link to the repository. Yuchen Jin reposted the news and commented "Real Open AI."

    Matched fieldsTitle

    #editorial#English text is being prepared.#Product updates#DeepSeek
  6. Nathan Lambert

    DeepSeek open-sources its inference framework DeepInfer

    DeepSeek announced that it is fully open-sourcing its self-developed inference framework DeepInfer, which is used to support large-scale, high-throughput, low-latency operation of DeepSeek-V4.1-Flash. The company said it is sharing its production stack to help the community lower compute costs and energy consumption and make AI more affordable. Nathan Lambert commented that this is a distinctive way to gain model adoption and is good for the community, and noted that the timing is delicate because the government has just banned DeepSeek in federal agencies.

    DeepSeek announced that it is fully open-sourcing its self-developed inference framework DeepInfer, which is used to support large-scale, high-throughput, low-latency operation of DeepSeek-V4.1-Flash. The company said it is sharing its production stack to help the community lower compute costs and energy consumption and make AI more affordable. Nathan Lambert commented that this is a distinctive way to gain model adoption and is good for the community, and noted that the timing is delicate becau

    Matched fieldsTitle

    #editorial#English text is being prepared.#DeepSeek#English text is being prepared.
  7. 36Kr · AISelected

    DeepSeek open-sources inference framework DeepInfer and Token codec component deepseek-recipe

    DeepSeek open-sources its self-developed inference framework DeepInfer (Apache 2.0) and the production-grade Token codec component deepseek-recipe (MIT), supporting DeepSeek V4.1 Flash in serving 155 trillion Tokens per day on a single cluster. Official online statistics show an average of 77K Tokens input per request, a time to first token of about 400 milliseconds, 230 Tokens output per second during the generation phase, and an average of 96% of input hitting the cache. DeepInfer currently supports V4.1 Flash; the open-source version provides only offline inference, and the KV cache management and load balancing code has not yet been fully open-sourced because it is tied to internal infrastructure.

    DeepSeek open-sources its self-developed inference framework DeepInfer (Apache 2.0) and the production-grade Token codec component deepseek-recipe (MIT), supporting DeepSeek V4.1 Flash in serving 155 trillion Tokens per day on a single cluster. Official online statistics show an average of 77K Tokens input per request, a time to first token of about 400 milliseconds, 230 Tokens output per second during the generation phase, and an average of 96% of input hitting the cache. DeepInfer currently su

    Matched fieldsTitle

    #hot-signal#English text is being prepared.#Reasoning#English text is being prepared.
  13. Ma Dongxi NLP

    DeepSeek Open-Sources Large-Scale Inference Framework DeepInfer Under Apache-2.0 License

    DeepSeek has open-sourced its high-performance large-scale inference framework DeepInfer under the Apache-2.0 license. The framework supports optimized kernels, model and sampler runtime, parallelism strategies such as Attention DP and balanced CP, MoE EP plus EPLB, as well as DSpark and confidence-scheduled verification, SWA bounded replay, asynchronous scheduling with a zero-overhead batch scheduler, batch-invariant inference, deterministic sampling, and structured output. Ma Dongxi NLP reposted the news and said "Respect."

    DeepSeek has open-sourced its high-performance large-scale inference framework DeepInfer under the Apache-2.0 license. The framework supports optimized kernels, model and sampler runtime, parallelism strategies such as Attention DP and balanced CP, MoE EP plus EPLB, as well as DSpark and confidence-scheduled verification, SWA bounded replay, asynchronous scheduling with a zero-overhead batch scheduler, batch-invariant inference, deterministic sampling, and structured output. Ma Dongxi NLP repost

    Matched fieldsTitle

    #hot-signal#English text is being prepared.#DeepSeek#Product updates
  14. Shao MengSelected

    DeepSeek Open-Sources High-Performance Inference Framework DeepInfer

    DeepSeek has open-sourced DeepInfer, the high-performance inference framework supporting its online services, under the Apache 2.0 license. This release focuses on offline inference, is built primarily on vLLM, and draws on SGLang and TensorRT-LLM. It integrates previously separately open-sourced kernels such as FlashMLA, DeepGEMM, DeepEP, and EPLB into a complete engine, filling in system layers such as scheduling, parallelism, and speculative decoding, while supporting NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. The original article also breaks down designs including DSpark speculative decoding, YoCo SWA bounded replay, asynchronous scheduling, batch-invariant deterministic inference, and regex/JSON schema/EBNF structured output.

    DeepSeek has open-sourced DeepInfer, the high-performance inference framework supporting its online services, under the Apache 2.0 license. This release focuses on offline inference, is built primarily on vLLM, and draws on SGLang and TensorRT-LLM. It integrates previously separately open-sourced kernels such as FlashMLA, DeepGEMM, DeepEP, and EPLB into a complete engine, filling in system layers such as scheduling, parallelism, and speculative decoding, while supporting NVIDIA SM100 GPUs and Hu

    Matched fieldsTitle

    #hot-signal#English text is being prepared.#Reasoning#English text is being prepared.
  15. Tianyi CuiSelected

    DeepSeek open-sources the large-scale inference framework DeepInfer under the Apache-2.0 license

    DeepSeek has open-sourced the high-performance large-scale inference framework DeepInfer under the Apache-2.0 license. The framework supports optimized kernels, model and sampler runtimes; on parallelism it provides Attention DP, balanced CP, and MoE EP + EPLB, and includes DSpark and confidence scheduling validation, SWA bounded replay, asynchronous scheduling of a zero-overhead batch scheduler, batch-invariant inference, deterministic sampling, and structured output.

    Matched fieldsTitle

    #hot-signal#English text is being prepared.#Reasoning#English text is being prepared.