Meta says it is training the Muse foundation model on the Prometheus supercomputing cluster

Meta announced that the first phase of its gigawatt-scale supercomputing cluster Prometheus has been completed in New Albany, Ohio, and is being used to train the Muse series of foundation models, including Muse Wonder. The cluster currently consists of hundreds of thousands of GPUs, with a full-scale target of about 1 million, and has deployed two generations of platforms, NVIDIA GB200 NVL72 (Catalina) and GB300 NVL72 (Clemente), with Vera Rubin NVL72 to be introduced later. Meta says Muse Wonder's pre-training goodput exceeds 85%, reinforcement learning goodput exceeds 90%, and the interruption rate has been reduced by 99% from its mid-2025 peak. Subsequent reports added that during Muse Wonder training, an interruption occurred on average only about every 47K GPU hours; Meta said that in a cluster of this scale, even a single host failing to meet performance standards would slow down an entire training round, so Prometheus was designed for large-scale fault tolerance.

AI-generated from public reports

1. Oct 20· 2 reports

   English translation is being prepared.

   Source:Meta Engineering Blog