Understand this story
On November 5, 2026, LMSYS published a blog post stating that it built a megakernel for Muse Spark Open from scratch using CUDA C++ and PTX, raising decoding speed on the GB300 from 517 tok/s to 1,109 tok/s. The kernel merges the attention and MoE of each layer into a single launch, and hides latency through weight prefetching, instruction cache warming, and fence-free cross-card data transfer. Currently, this implementation only supports speculative decoding with a batch size of 1, and the code has been open-sourced at facebookresearch/sgl-muse-mega.
AI-generated from public reports
Latest development · Report
LMSYS hand-writes a megakernel for Muse Spark Open, boosting decoding speed from 517 to 1,109…Developments
1 confirmed developments
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 5
LMSYS BlogFirst-partySelected
LMSYS hand-writes a megakernel for Muse Spark Open, boosting decoding speed from 517 to 1,109…
LMSYS built a megakernel from scratch for Muse Spark Open using CUDA C++ and PTX, raising decoding speed from 517 tok/s to 1,109 tok/s on GB300. The kernel merges each layer's attention and MoE into a single launch, hiding latency through weight prefetching, instruction cache warming, and fence-free cross-GPU data transfer. It currently only supports speculative decoding with a batch size of 1, and the code is open-sourced at facebookresearch/sgl-muse-mega.