Understand this story
On November 9, 2026, Google Cloud announced that its open-source JAX training framework MaxText added long-context training support in v0.2.4, providing four context parallelism strategies on Cloud TPU: All-gather, A2A (DeepSpeed Ulysses), P2P (Ring Attention), and A2A+P2P hybrid (USP), all based on Tokamax's Splash Attention kernel. In benchmarks, Llama 3.1 8B achieved 70% MFU on TPU 5p at sequence lengths up to 4 million tokens, and both TPU 5p and TPU 7 can scale to 14 million tokens. This feature has been released with MaxText v0.2.4 on PyPI, Docker images, and GitHub main.
AI-generated from public reports
Latest development · Report
Google Cloud's open source framework MaxText supports long-context training on TPUDevelopments
1 confirmed developments
- Source:Google Developers Blog
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 9
Google Developers BlogFirst-party
Google Cloud's open source framework MaxText supports long-context training on TPU
Google Cloud's open source JAX training framework MaxText adds long-context training support in v0.2.4, providing four context parallelism strategies on Cloud TPU: All-gather, A2A (DeepSpeed Ulysses), P2P (Ring Attention), and A2A+P2P hybrid (USP), all based on Tokamax's Splash Attention kernel. In benchmarks, Llama 3.1 8B on TPU 5p reaches 70% MFU at sequence lengths up to 4 million tokens, and both TPU 5p and TPU 7 can scale to 14 million tokens. This feature has been released with MaxText v0.2.4 on PyPI, Docker images, and GitHub main.