Understand this story
On September 30, 2026, Fireworks published a GLM 5.2 numerical alignment experiment. The experiment shows that when running the same reinforcement learning task on GLM 5.2, an inconsistency between the trainer and the rollout engine in how token probabilities are computed leads to a KL of about 0.013, roughly 45% of samples discarded per batch, and the reward dropping from about 0.9 to below 0.2 around step 20; after numerical alignment, the KL is 0 and the reward remains stable throughout. The same material also documents a Qwen3.5-MoE experiment: precision differences in the weighted sum of expert outputs brought the k3 of image tokens to 0.296, about 300 times the equivalence threshold, and k3 dropped to zero after merely replacing the MoE module with the Hugging Face reference implementation. Currently, this experiment is published by Fireworks as a first-party source, and no subsequent independent progress has emerged.
AI-generated from public reports
Latest development · Report
Reinforcement Learning: Why Numerical Alignment and MoE Routing Matter So MuchDevelopments
1 confirmed developments
- Source:Fireworks AI
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Sep 30
Fireworks AIFirst-party
Reinforcement Learning: Why Numerical Alignment and MoE Routing Matter So Much
Fireworks experiments show that when running the same RL task on GLM 5.2, inconsistent token probability computation between the trainer and the rollout engine leads to a KL of about 0.013, about 45% of samples discarded per batch, and reward dropping from about 0.9 to below 0.2 around step 20; after numerical alignment, KL is 0 and reward remains stable throughout. In the Qwen3.5-MoE experiment, precision differences in the weighted sum of expert outputs push the image token k3 to 0.296, about 300 times the equivalence threshold; replacing only the MoE module with the Hugging Face reference implementation brings k3 to zero.