百度开源 ERNIE 5.5 Flash Base:280B MoE 模型,激活参数 14B,支持 1M 上下文
Stay tuned for more!
百度开源 ERNIE 5.5 Flash Base,这是一个 280B 参数的 MoE 模型,激活参数仅 14B,预训练使用 33.5T tokens。该模型在推理、编码和长上下文基准上表现较强,支持最长 1M 上下文,后训练已在进行中。百度官方账号转发该发布并表示后续还有更多内容。
查看原帖很激动地介绍 ERNIE 5.5 Flash Base,这是我加入 @Baidu_Inc 以来的首个发布! 一个开源的 280B MoE 模型,激活参数仅 14B,预训练使用 33.5T tokens。它在推理、编码和长上下文基准上表现强劲,支持最长 1M 上下文。 向背后出色的团队致以崇高敬意和感谢——工作非常出色! 后训练已经启动,我迫不及待想尽快把它带给你们。 而这仅仅是开始——更多令人兴奋的技术进展正在路上。敬请期待! https://t.co/7DBNYNwf81
查看原文
Excited to introduce ERNIE 5.5 Flash Base, my first release since joining @Baidu_Inc ! An open-sourced 280B MoE model with just 14B active parameters, pretrained on 33.5T tokens. It delivers strong performance across reasoning, coding, and long-context benchmarks, with support for up to 1M context. Huge respect and thanks to the incredible team behind this — outstanding work! Post-training is already underway, and I can't wait to bring it to you soon. And this is just the beginning — more exciting technical developments are on the way. Stay tuned! https://t.co/7DBNYNwf81
正文
Stay tuned for more!