←Search results

Search “Reasoning”

By relevance · Total 3162 results Clear search

Friday

  1. Story1 reports · 1 facts

    Lenovo team tests inference performance of compute-storage fusion pooled system

    The Lenovo team tested the inference performance of a compute-storage fusion pooled system. Xiao Limin, Chief Scientist of Lenovo Group, proposed a compute-storage fusion pooled system that combines memory pooling with near-memory computing, targeting two major pain points: insufficient KV Cache capacity and high GPU memory access overhead. Tests show that memory pool offloading reduces first-token latency by 3.5x, KV cross-instance sharing achieves up to 12~28x speedup compared with full recomputation, offloading attention computation reduces decoding latency by 2.5x, and end-to-end throughput improves by 6~20x as context length grows. The team's next step is to evolve from G2.5 to G1.5 with the help of supernode scale-up interconnect.

    The Lenovo team tested the inference performance of a compute-storage fusion pooled system. Xiao Limin, Chief Scientist of Lenovo Group, proposed a compute-storage fusion pooled system that combines memory pooling with near-memory computing, targeting two major pain points: insufficient KV Cache capacity and high GPU memory access overhead. Tests show that memory pool offloading reduces first-token latency by 3.5x, KV cross-instance sharing achieves up to 12 28x speedup compared with full recomp

    Matched fieldsTitle

  2. FactRelated story

    Lenovo team tests inference performance of compute-storage fusion pooled system

    The Lenovo team tested the inference performance of a compute-storage fusion pooled system. Xiao Limin, Chief Scientist of Lenovo Group, proposed a compute-storage fusion pooled system that combines memory pooling with near-memory computing, targeting two major pain points: insufficient KV Cache capacity and high GPU memory access overhead. Tests show that memory pool offloading reduces first-token latency by 3.5x, KV cross-instance sharing achieves up to 12~28x speedup compared with full recomputation, offloading attention computation reduces decoding latency by 2.5x, and end-to-end throughput improves by 6~20x as context length grows. The team's next step is to evolve from G2.5 to G1.5 with the help of supernode scale-up interconnect.

    Matched fieldsTitle

    View story:Lenovo team tests inference performance of compute-storage fusion pooled system

  3. Story1 reports · 1 facts

    MiniMax and Inferact collaborate to advance production inference for models

    On October 23, 2026, MiniMax announced a partnership with Inferact to jointly advance production inference for MiniMax models. MiniMax stated that the relevant engineering improvements have been contributed back to the vLLM project, and thanked the Inferact team for supporting M3.1 Flash in the production environment from day one. Currently, this partnership was officially announced by MiniMax, and no further progress has been disclosed.

    Matched fieldsTitle

  4. FactRelated story

    MiniMax partners with Inferact to advance production inference for models

    On October 23, 2026, MiniMax announced a partnership with Inferact to jointly advance production inference for MiniMax models. MiniMax stated that the relevant engineering improvements have been contributed back to the vLLM project, and thanked the Inferact team for supporting M3.1 Flash in the production environment from day one. Currently, this partnership was officially announced by MiniMax, and no further progress has been disclosed.

    Matched fieldsTitle

    View story:MiniMax and Inferact collaborate to advance production inference for models

Thursday

  1. Story1 reports · 1 facts

    MiniMax thanks for bringing H3 to TPU and looks forward to open-source work on efficient video inference

    MiniMax responded to SGLang including MiniMax H3 in the TPU support list of SGLang-TorchTPU, expressing thanks and saying it looks forward to more open-source work around efficient video inference. SGLang said that SGLang-TorchTPU was jointly built by SGLang, the Google TPU team, and the Meta PyTorch team, has been used for Meta's production inference, and supports models including Muse Spark Open, GLM-5.3 Flash, Kimi K3, Ling 3.1 Flash, and MiniMax H3. There are currently no facts about subsequent developments.

    MiniMax responded to SGLang including MiniMax H3 in the TPU support list of SGLang-TorchTPU, expressing thanks and saying it looks forward to more open-source work around efficient video inference. SGLang said that SGLang-TorchTPU was jointly built by SGLang, the Google TPU team, and the Meta PyTorch team, has been used for Meta's production inference, and supports models including Muse Spark Open, GLM-5.3 Flash, Kimi K3, Ling 3.1 Flash, and MiniMax H3. There are currently no facts about subsequ

    Matched fieldsTitle

  2. FactRelated story

    MiniMax thanks for bringing H3 to TPU and looks forward to efficient video inference open-source work

    MiniMax responded to SGLang including MiniMax H3 in the TPU support list of SGLang-TorchTPU, expressing thanks and saying it looks forward to more open-source work around efficient video inference. SGLang said that SGLang-TorchTPU was jointly built by SGLang, the Google TPU team, and the Meta PyTorch team, has been used for Meta's production inference, and supports models including Muse Spark Open, GLM-5.3 Flash, Kimi K3, Ling 3.1 Flash, and MiniMax H3. There are currently no facts about subsequent developments.

    Matched fieldsTitle

    View story:MiniMax thanks for bringing H3 to TPU and looks forward to open-source work on efficient video inference

Tuesday

  1. Story25 reports · 2 facts

    DeepSeek open-sources inference framework DeepInfer

    On October 20, 2026, DeepSeek announced the open-sourcing of its internal production-grade inference framework, DeepInfer. According to Latent Space, the framework powers DeepSeek-V4.1-Flash, is customized based on vLLM, supports distributed inference, speculative decoding, SWA Bounded Replay, and structured output, and targets the NVIDIA SM100 and Huawei Ascend 950 platforms; the published single-cluster production figures are 155 trillion tokens processed per day, a 96% input cache hit rate, 400 ms first-token latency, and a generation speed of 230 tokens/s. The report also noted that components such as KV cache management and load balancing have not yet been fully open-sourced. The next day, DeepSeek published the DeepInfer technical report on Zhihu, disclosing internal details including the hardware selection for Prefill and Decode and the KVCache targets, and providing data showing a single cluster serving 155 trillion Tokens/day at $38 per card per hour at off-peak prices. The framework is currently open-sourced, but the KV cache system, Prefill/Decode disaggregation, and load balancing required for a complete service are not fully included in this open-source project.

    On October 20, 2026, DeepSeek announced the open-sourcing of its internal production-grade inference framework, DeepInfer. According to Latent Space, the framework powers DeepSeek-V4.1-Flash, is customized based on vLLM, supports distributed inference, speculative decoding, SWA Bounded Replay, and structured output, and targets the NVIDIA SM100 and Huawei Ascend 950 platforms; the published single-cluster production figures are 155 trillion tokens processed per day, a 96% input cache hit rate, 4

    Matched fieldsTitle

  2. FactRelated story

    DeepSeek open-sources inference framework DeepInfer

    On October 20, 2026, DeepSeek announced the open-sourcing of its internal production-grade inference framework, DeepInfer. According to Latent Space, the framework powers DeepSeek-V4.1-Flash, is customized based on vLLM, supports distributed inference, speculative decoding, SWA Bounded Replay, and structured output, and targets the NVIDIA SM100 and Huawei Ascend 950 platforms; the published single-cluster production figures are 155 trillion tokens processed per day, a 96% input cache hit rate, 400 ms first-token latency, and a generation speed of 230 tokens/s. The report also noted that components such as KV cache management and load balancing have not yet been fully open-sourced. The next day, DeepSeek published the DeepInfer technical report on Zhihu, disclosing internal details including the hardware selection for Prefill and Decode and the KVCache targets, and providing data showing a single cluster serving 155 trillion Tokens/day at $38 per card per hour at off-peak prices. The framework is currently open-sourced, but the KV cache system, Prefill/Decode disaggregation, and load balancing required for a complete service are not fully included in this open-source project.

    Matched fieldsTitle

    View story:DeepSeek open-sources inference framework DeepInfer

  3. FactRelated story

    DeepSeek open-sources inference framework DeepInfer

    On October 20, 2026, DeepSeek open-sourced its internal inference framework DeepInfer. According to Yuchen Jin, citing data disclosed by DeepSeek, a single cluster processes 155 trillion tokens per day and generates $38 in revenue per GPU-hour (off-peak pricing); DeepInfer supports both NVIDIA Blackwell and Huawei Ascend. A subsequent 36Kr report stated that its accompanying open-source deepseek-recipe supports NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. On October 21, swyx, citing Latent.Space data, said a single cluster processes 155T tokens per day with a 96% input cache hit rate, and argued that the lifetime value of each sold GPU has risen by about 30%. On October 24, Nathan Lambert said DeepSeek's single cluster processes 150T tokens per day, exceeding the combined open-source model inference volume of OpenRouter, Fireworks, Together, and BaseTen, and noted that the actual number of clusters is unknown; on the same day, LMSYS published a same-hardware benchmark comparison, stating that SGLang meets or exceeds DeepInfer on DeepInfer's own benchmark, with prefill throughput up to 1.04x and decode step length shortened by 1.1%–7.4% on 64x GB300.

    Matched fieldsTitle

    View story:DeepSeek open-sources inference framework DeepInfer

  4. FactRelated story

    Zyphra adds second round of inference optimization to ZOE

    On October 20, 2026, Zyphra released the second generation of inference optimizations, ZOE, adding it to the ZOE inference optimization series. According to information published by Zyphra Research, on AMD Instinct MI355X, GLM-5.3's time to first token (TTFT) at 512K context dropped from 178.8 seconds to 64.6 seconds, about 2.8x; at 90% prefix cache hit rate, it dropped from 42.3 seconds to 11.8 seconds, about 3.6x. Its CSA top-K selection HIP kernel raised DeepSeek-V4-Pro's single-user decoding throughput at 1M context from 43.5 to 61.1 tokens/s, about 1.40x. GLM-5.3's AMD MXFP4 quantization compressed the model from 756 GB to 410 GB, with benchmark scores such as AIME25 within 3 points of FP8. Currently this release is the only development with evidence, and there are no subsequent developments yet.

    Matched fieldsTitle

    View story:Zyphra adds second round of inference optimizations to ZOE

Wednesday

Sunday

  1. Story1 reports · 1 facts

    Zyphra adds second round of inference optimizations to ZOE

    On October 20, 2026, Zyphra released the second generation of inference optimizations, ZOE, adding it to the ZOE inference optimization series. According to information published by Zyphra Research, on AMD Instinct MI355X, GLM-5.3's time to first token (TTFT) at 512K context dropped from 178.8 seconds to 64.6 seconds, about 2.8x; at 90% prefix cache hit rate, it dropped from 42.3 seconds to 11.8 seconds, about 3.6x. Its CSA top-K selection HIP kernel raised DeepSeek-V4-Pro's single-user decoding throughput at 1M context from 43.5 to 61.1 tokens/s, about 1.40x. GLM-5.3's AMD MXFP4 quantization compressed the model from 756 GB to 410 GB, with benchmark scores such as AIME25 within 3 points of FP8. Currently this release is the only development with evidence, and there are no subsequent developments yet.

    On October 20, 2026, Zyphra released the second generation of inference optimizations, ZOE, adding it to the ZOE inference optimization series. According to information published by Zyphra Research, on AMD Instinct MI355X, GLM-5.3's time to first token (TTFT) at 512K context dropped from 178.8 seconds to 64.6 seconds, about 2.8x; at 90% prefix cache hit rate, it dropped from 42.3 seconds to 11.8 seconds, about 3.6x. Its CSA top-K selection HIP kernel raised DeepSeek-V4-Pro's single-user decoding

    Matched fieldsTitle