Understand this story
On October 20, 2026, Google released a TorchTitan branch adapted for Google Cloud TPUs. According to the Google Developers Blog, Google integrated TorchTPU into Meta's TorchTitan, and Cloud TPUs can now directly train PyTorch models through the native backend torch.device("tpu") without rewriting them into JAX. The TPU branch provides pretraining recipes for Llama 3 70B/405B, Qwen 3 8B/32B/235B MoE, and FLUX.1 12B, and replaces standard implementations with Pallas kernels such as Splash Attention. On FLUX, SimpleFSDP enables TorchTitan to reach performance parity of 99% with JAX MaxDiffusion. There are currently no subsequent development facts.
AI-generated from public reports
Latest development · Report
Google integrates TorchTPU into TorchTitan to optimize model training for TPUsDevelopments
1 confirmed developments
- Source:Google Developers Blog
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 20
Google Developers BlogFirst-party
Google integrates TorchTPU into TorchTitan to optimize model training for TPUs
Google integrates TorchTPU into Meta's TorchTitan; Cloud TPU can now train PyTorch models directly through the native backend torch.device("tpu"), without rewriting them into JAX. The TorchTitan TPU branch provides pretraining recipes for Llama 3 70B/405B, Qwen 3 8B/32B/235B MoE, and FLUX.1 12B, and replaces standard implementations with Pallas kernels such as Splash Attention. On FLUX, SimpleFSDP brings TorchTitan to 99% performance parity with JAX MaxDiffusion.