Prime Intellect releases the Prime Mega MoE training kernel
Prime Intellect releases the Prime Mega MoE training kernel. This kernel builds on the Mega MoE kernel from DeepSeek's open-source DeepGEMM library by adding a fused and overlapped backward pass, making it usable for training rather than inference only. In the prime-rl trainer, a full training step for the Qwen3 MoE model is 32% to 34% faster than the baseline and 48% to 55% faster than TorchTitan; single-layer forward pass speedup reaches up to 3.11x (from 12.0 ms to 3.9 ms), but the backward pass for Qwen3-235B-A22B is only 1.24x faster. The kernel supports BF16, MXFP8 on Blackwell, and blockwise FP8 on Hopper, is released under Apache 2.0 with prime-rl, and is enabled via the mega_moe dispatch backend.
Prime Intellect releases the Prime Mega MoE training kernel. This kernel builds on the Mega MoE kernel from DeepSeek's open-source DeepGEMM library by adding a fused and overlapped backward pass, making it usable for training rather than inference only. In the prime-rl trainer, a full training step for the Qwen3 MoE model is 32% to 34% faster than the baseline and 48% to 55% faster than TorchTitan; single-layer forward pass speedup reaches up to 3.11x (from 12.0 ms to 3.9 ms), but the backward p
Matched fieldsSummary