Understand this story
On November 16, 2026, Google open-sourced the AI-Hypercomputer/tpu-codesign guide, which sets out model design and optimization rules for Ironwood (TPU v7x) and subsequent TPUs, covering shape alignment, architectural choices such as LatentMoE, and parallelization strategies. The guide requires dimensions such as d_model, FFN width, and head_dim to be aligned to 256; in testing, changing the Qwen3-30B MoE attention from 32×128 to 16×256 sped up training, and the longer the context, the greater the benefit. The same principles also apply to the TPU 8t and TPU 8i, which newly add native FP4. There are currently no further developments.
AI-generated from public reports
Latest development · Report
Google releases TPU model design and optimization guideDevelopments
1 confirmed developments
Google releases open-source guide for TPU model design and optimization
Source:Google Developers Blog
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 16
Google Developers BlogFirst-party
Google releases TPU model design and optimization guide
Google has open-sourced the AI-Hypercomputer/tpu-codesign guide, which provides model design rules for Ironwood (TPU v7x) and subsequent TPUs, covering shape alignment, architectural choices such as LatentMoE, and parallelization strategies. The guide requires aligning dimensions such as d_model, FFN width, and head_dim to 256; in testing, changing the Qwen3-30B MoE attention from 32×128 to 16×256 sped up training, with greater gains as context length increases. The same principles also apply to the TPU 8t and TPU 8i, which add native FP4.