AI News

Trending story

Google implements Zero-TIM training-inference numerical alignment in Tunix

1 reports1 story updatesUpdated Nov 16 08:00

Understand this story

AI overview

The Google Developers Blog introduces that Google has implemented the Zero-TIM zero training-inference mismatch scheme in Tunix, achieving bit-for-bit identical logprobs for the same token between the sampler and the trainer on 64 TPU v5p chips (DP8×TP8), eliminating the numerical discrepancy between training and inference. The specific approach includes fixing the accumulation order within shards, explicitly specifying the reduction order across shards, unifying the logprob computation path for sampling actions, and writing custom differentiation rules for the serving-side kernel. Validated on FrozenLake with Qwen3-8B over 200 steps, Zero-TIM's sampler and trainer logprobs show a mean absolute difference of zero, and the observed solve rate is higher than that of standard training and TIS.

AI-generated from public reports

Latest development · Report

Google uses Zero-TIM on TPUs to eliminate numerical discrepancies between training and inference

Developments

1 confirmed developments

  1. Google implements Zero-TIM training-inference numerical alignment in Tunix

    Source:Google Developers Blog

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 16

  1. Google Developers BlogFirst-party

    Google uses Zero-TIM on TPUs to eliminate numerical discrepancies between training and inference

    The Google Developers Blog introduces Zero-TIM, implemented in Tunix, which on 64 TPU v5p chips (DP8×TP8) makes the sampler and trainer produce bitwise-identical logprobs for the same token, eliminating numerical discrepancies between training and inference. The approach includes fixing the accumulation order within shards, explicitly specifying the reduction order across shards, unifying the logprob computation path for sampling actions, and writing custom differentiation rules for the server-side kernel. Validated on FrozenLake with Qwen3-8B over 200 steps, Zero-TIM's sampler and trainer logprobs have a mean absolute difference of zero, and the observed solve rate is higher than that of standard training and TIS.

Related stories

  1. Related storiesApple research team submits paper to arXiv disclosing the HALO system
  2. Related storiesDeepSeek open-sources inference framework DeepInfer
  3. Related storiesNvidia releases Nemotron 3.5 Super VL and Bolt models
  4. Related storiesEnglish translation is being prepared.