English translation is being prepared.
Matched fieldsSummary
←Search results
Search “Reasoning”
English translation is being prepared.
Matched fieldsSummary
English translation is being prepared.
Matched fieldsSummary
English translation is being prepared.
Matched fieldsSummary
English translation is being prepared.
Matched fieldsSummary
English translation is being prepared.
Matched fieldsSummary
Hao AI Lab open-sources the JetSpec++ model, code, and inference engine
On October 31, 2026, the UCSD Hao AI Lab announced that it would fully open-source the JetSpec++ model, code, and inference engine, along with links to the paper, code, and Hugging Face. This speculative decoding framework pipelines draft generation and verification through causal parallel drafting, a hidden-state predictor, and an anchor-token decoder. According to the data it published, on Qwen3-8B, JetSpec++ achieves 1,361 TPS, delivering up to 1.54x end-to-end speedup relative to DSpark and 7.45x relative to autoregressive decoding, while remaining lossless; after combining NVFP4 with kernel optimizations, on two GB200 GPUs, Qwen3-8B exceeds 4,000 TPS and Qwen3.8-27B exceeds 900 TPS. There are currently no further developments.
On October 31, 2026, the UCSD Hao AI Lab announced that it would fully open-source the JetSpec++ model, code, and inference engine, along with links to the paper, code, and Hugging Face. This speculative decoding framework pipelines draft generation and verification through causal parallel drafting, a hidden-state predictor, and an anchor-token decoder. According to the data it published, on Qwen3-8B, JetSpec++ achieves 1,361 TPS, delivering up to 1.54x end-to-end speedup relative to DSpark and
Matched fieldsTitle
Hao AI Lab open-sources the JetSpec++ model, code, and inference engine
On October 31, 2026, the UCSD Hao AI Lab announced that it would fully open-source the JetSpec++ model, code, and inference engine, along with links to the paper, code, and Hugging Face. This speculative decoding framework pipelines draft generation and verification through causal parallel drafting, a hidden-state predictor, and an anchor-token decoder. According to the data it published, on Qwen3-8B, JetSpec++ achieves 1,361 TPS, delivering up to 1.54x end-to-end speedup relative to DSpark and 7.45x relative to autoregressive decoding, while remaining lossless; after combining NVFP4 with kernel optimizations, on two GB200 GPUs, Qwen3-8B exceeds 4,000 TPS and Qwen3.8-27B exceeds 900 TPS. There are currently no further developments.
Matched fieldsTitle
View story:Hao AI Lab open-sources the JetSpec++ model, code, and inference engine
DeepSeek open-sources inference framework DeepInfer
On October 20, 2026, DeepSeek announced the open-sourcing of its internal production-grade inference framework, DeepInfer. According to Latent Space, the framework powers DeepSeek-V4.1-Flash, is customized based on vLLM, supports distributed inference, speculative decoding, SWA Bounded Replay, and structured output, and targets the NVIDIA SM100 and Huawei Ascend 950 platforms; the published single-cluster production figures are 155 trillion tokens processed per day, a 96% input cache hit rate, 400 ms first-token latency, and a generation speed of 230 tokens/s. The report also noted that components such as KV cache management and load balancing have not yet been fully open-sourced. The next day, DeepSeek published the DeepInfer technical report on Zhihu, disclosing internal details including the hardware selection for Prefill and Decode and the KVCache targets, and providing data showing a single cluster serving 155 trillion Tokens/day at $38 per card per hour at off-peak prices. The framework is currently open-sourced, but the KV cache system, Prefill/Decode disaggregation, and load balancing required for a complete service are not fully included in this open-source project.
On October 20, 2026, DeepSeek announced the open-sourcing of its internal production-grade inference framework, DeepInfer. According to Latent Space, the framework powers DeepSeek-V4.1-Flash, is customized based on vLLM, supports distributed inference, speculative decoding, SWA Bounded Replay, and structured output, and targets the NVIDIA SM100 and Huawei Ascend 950 platforms; the published single-cluster production figures are 155 trillion tokens processed per day, a 96% input cache hit rate, 4
Matched fieldsTitle
English translation is being prepared.
Matched fieldsTitle
English translation is being prepared.
Matched fieldsTitle
English translation is being prepared.
Matched fieldsTitle
English translation is being prepared.
Matched fieldsTitle
English translation is being prepared.
Matched fieldsTitle
English translation is being prepared.
Matched fieldsTitle
English translation is being prepared.
Matched fieldsTitle