Meta 在 Prometheus 超算训练 Muse 基础模型，预训练 goodput 超 85%
We trained our Muse foundational models, including Muse Wonder, at Prometheus, our gigawatt-scale supercluster in New Albany, Ohio.
Meta 在位于俄亥俄州 New Albany 的吉瓦级超算 Prometheus 上训练了包括 Muse Wonder 在内的 Muse 基础模型。该集群拥有数十万块 GPU，预训练 goodput 达 85% 以上，强化学习达 90% 以上，Muse Wonder 训练期间平均每约 47K GPU 小时才发生一次中断。Meta 表示，在如此规模的集群中，哪怕一台主机性能不达标也会拖慢整轮训练，因此 Prometheus 按大规模容错来设计。
推荐理由
AI 评分 52
Meta 公开了 Prometheus 超算的训练稳定性指标，可据此了解吉瓦级集群在预训练和强化学习中的实际运行效率。
主题
正文
当前显示原文，中文全文译文尚未就绪。
We trained our Muse foundational models, including Muse Wonder, at Prometheus, our gigawatt-scale supercluster in New Albany, Ohio.
Across a supercluster with hundreds of thousands of GPUs, even one underperforming host can slow an entire training run. We engineered Prometheus for resilience at scale, achieving 85%+ goodput on pretraining, 90%+ on reinforcement learning and ~47K GPU hours mean time between interruptions during Muse Wonder training.
Take a look at the tech powering Prometheus: https://t.co/WSFODnGRST