AI News

Trending story

LMSYS builds a megakernel for Muse Spark Open, decoding over a thousand tokens per second

1 reports1 story updatesUpdated Nov 5 08:00

Understand this story

AI overview

On November 5, 2026, LMSYS published a blog post stating that it built a megakernel for Muse Spark Open from scratch using CUDA C++ and PTX, raising decoding speed on the GB300 from 517 tok/s to 1,109 tok/s. The kernel merges the attention and MoE of each layer into a single launch, and hides latency through weight prefetching, instruction cache warming, and fence-free cross-card data transfer. Currently, this implementation only supports speculative decoding with a batch size of 1, and the code has been open-sourced at facebookresearch/sgl-muse-mega.

AI-generated from public reports

Latest development · Report

LMSYS hand-writes a megakernel for Muse Spark Open, boosting decoding speed from 517 to 1,109…

Developments

1 confirmed developments

  1. LMSYS builds megakernel for Muse Spark Open achieving over a thousand tokens per second

    Source:LMSYS Blog

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 5

  1. LMSYS BlogFirst-partySelected

    LMSYS hand-writes a megakernel for Muse Spark Open, boosting decoding speed from 517 to 1,109…

    LMSYS built a megakernel from scratch for Muse Spark Open using CUDA C++ and PTX, raising decoding speed from 517 tok/s to 1,109 tok/s on GB300. The kernel merges each layer's attention and MoE into a single launch, hiding latency through weight prefetching, instruction cache warming, and fence-free cross-GPU data transfer. It currently only supports speculative decoding with a batch size of 1, and the code is open-sourced at facebookresearch/sgl-muse-mega.

Related stories

  1. Related storiesFireworks publishes Kimi K3 reinforcement learning infrastructure report
  2. Related storiesEnglish translation is being prepared.
  3. Related storiesEnglish translation is being prepared.
  4. Related storiesEnglish translation is being prepared.