Understand this story
On November 4, 2026, LMSYS released XM-Sparse, which integrates Netpreme X-MemTM with SGLang HiSparse to store offloaded KV cache in high-bandwidth external memory. According to results published on the LMSYS Blog, with a 16K context, 20% top-k miss, and a per-GPU batch of 192, the average time per output token (TPOT) is 49.25% lower than host DRAM over PCIe, and 63.58% lower at 40% miss; with 16K–1M contexts and 20% miss, peak decoding throughput is 2.09–3.04 times that of host DRAM. These results were measured on the X-MemTM Pilot virtual machine; the X-MemTM chip will be launched later. There are currently no further developments.
AI-generated from public reports
Latest development · Report
XM-Sparse: How Netpreme X-Mem™ Pushes Sparse Attention Throughput to New HeightsDevelopments
1 confirmed developments
- Source:LMSYS Blog
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 4
LMSYS BlogFirst-party
XM-Sparse: How Netpreme X-Mem™ Pushes Sparse Attention Throughput to New Heights
LMSYS integrated Netpreme X-Mem™ with SGLang HiSparse to store offloaded KV cache in high-bandwidth external memory. At a 16K context, 20% top-k miss, and per-GPU batch of 192, the average time per output token (TPOT) was reduced by 49.25% compared with host DRAM over PCIe, and by 63.58% at 40% miss; at 16K–1M context and 20% miss, peak decoding throughput was 2.09–3.04 times that of host DRAM. Results were measured on the X-Mem™ Pilot virtual machine; the X-Mem™ chip will launch later.