Understand this story
November 19, 2026—Streaming DiLoCo is now available through the PyPI release of MaxText v0.2.5, the corresponding Docker image, and the GitHub main branch. Google implemented the method in MaxText to support distributed large language model training across data centers with low communication overhead: each learner runs locally for 37 to 65 steps before global synchronization, while the parameters are divided into NLAYERS+1 segments and transmitted in an interleaved manner, reducing communication frequency by a factor of 37 to 65 while maintaining convergence and model quality comparable to standard synchronous training. There are currently no other factually substantiated developments.
AI-generated from public reports
Latest development · Report
Google Implements Streaming DiLoCo in MaxText for Cross-Region Distributed LLM TrainingDevelopments
1 confirmed developments
- Source:Google Developers Blog
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 19
Google Developers BlogFirst-party
Google Implements Streaming DiLoCo in MaxText for Cross-Region Distributed LLM Training
Google has implemented Streaming DiLoCo in MaxText to support distributed LLM training across data centers with low communication overhead. Each learner runs locally for H=37–65 steps before global synchronization, while the parameters are divided into P=NLAYERS+1 segments and transmitted in an interleaved manner, reducing communication frequency by a factor of H while maintaining convergence and model quality comparable to standard synchronous training.
AI Trends