Understand this story
The English summary is being updated. See the reporting timeline below.
Latest development · Report
SGLang Integrates Humming 0.1.18, Expanding Support for Low-Bit MoE InferenceDevelopments
1 confirmed developments
- Source:LMSYS Blog
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 18
LMSYS BlogFirst-party
SGLang Integrates Humming 0.1.18, Expanding Support for Low-Bit MoE Inference
SGLang has integrated Humming 0.1.18, bringing more quantization schemes and GPU architectures to its low-bit dense and MoE inference kernels while retaining the framework’s routing, scheduling, and distributed execution capabilities. When running DeepSeek-V4.1-Flash-1017 on eight H20-3e GPUs, Humming delivers a 1.07–1.23× prefill speedup over Marlin at the same A16 precision; after Humming EP1 switches to AFP8, decoding throughput increases by 16.79–34.41%. The article also explains the compute paths for Hopper and Blackwell, SGLang configuration, and related quality evaluation results.
AI Trends