NVIDIA uses a 12,000-line Megakernel to optimize Kimi K3 inference, and vLLM says AI code is now faster than human review
Benjamin Chislett, an engineer at NVIDIA CentML, used a 12,000-line Megakernel written in CuTe DSL to optimize Moonshot AI's Kimi K3 inference. The vLLM blog reports over 840 tokens/s at batch 1 and a 1.5x to 2x increase in decoding throughput across multiple configurations, while SemiAnalysis estimates 423 tok/s/user on GB300 NVL72, a 1.68x improvement. The vLLM blog says agent-optimized inference code is now outpacing human review and maintenance speed, so the team proposed a fast-track model optimization RFC that lets reviewers judge designs from high-level abstractions rather than reviewing code line by line. The article also mentions that NVIDIA is developing optimizations for other models for vLLM, such as DeepSeek V4.1.
Benjamin Chislett, an engineer at NVIDIA CentML, used a 12,000-line Megakernel written in CuTe DSL to optimize Moonshot AI's Kimi K3 inference. The vLLM blog reports over 840 tokens/s at batch 1 and a 1.5x to 2x increase in decoding throughput across multiple configurations, while SemiAnalysis estimates 423 tok/s/user on GB300 NVL72, a 1.68x improvement. The vLLM blog says agent-optimized inference code is now outpacing human review and maintenance speed, so the team proposed a fast-track model
Matched fieldsSummary、Body