AI Trends

Trending story

Streaming DiLoCo Is Now Available in MaxText v0.2.5

1 reports1 story updatesUpdated Nov 19 08:00

Understand this story

AI overview

November 19, 2026—Streaming DiLoCo is now available through the PyPI release of MaxText v0.2.5, the corresponding Docker image, and the GitHub main branch. Google implemented the method in MaxText to support distributed large language model training across data centers with low communication overhead: each learner runs locally for 37 to 65 steps before global synchronization, while the parameters are divided into NLAYERS+1 segments and transmitted in an interleaved manner, reducing communication frequency by a factor of 37 to 65 while maintaining convergence and model quality comparable to standard synchronous training. There are currently no other factually substantiated developments.

AI-generated from public reports

Latest development · Report

Google Implements Streaming DiLoCo in MaxText for Cross-Region Distributed LLM Training

Developments

1 confirmed developments

  1. Streaming DiLoCo is now available in MaxText v0.2.5

    Source:Google Developers Blog

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 19

  1. Google Developers BlogFirst-party

    Google Implements Streaming DiLoCo in MaxText for Cross-Region Distributed LLM Training

    Google has implemented Streaming DiLoCo in MaxText to support distributed LLM training across data centers with low communication overhead. Each learner runs locally for H=37–65 steps before global synchronization, while the parameters are divided into P=NLAYERS+1 segments and transmitted in an interleaved manner, reducing communication frequency by a factor of H while maintaining convergence and model quality comparable to standard synchronous training.

Related stories

  1. Related storiesGoogle launches Gemini 4 Hydrogen on Antigravity
  2. Related storiesMeta launches Muse Wonder and Muse Wonder Full models
  3. Related storiesGoogle supports long-context training in MaxText
  4. Related storiesGoogle open-sources TorchTPU native PyTorch stack