Moonshot AI 开源 Kimi K2:1T 参数 MoE 智能体模型
Moonshot AI 开源 Kimi K2,这是总参数 1 万亿、激活参数 320 亿的 MoE 模型,包含基础版 Kimi-K2-Base 和后训练版 Kimi-K2-Instruct,在非思考类模型的前沿知识、数学和编码任务上达到 SOTA。该模型针对智能体任务优化,可自动调用工具完成多步任务,并采用 MuonClip 优化器在 15.5T token 上预训练且无训练尖峰。Kimi 网页端和移动端用户即日起可免费选用,API 兼容 OpenAI/Anthropic 接口,暂不支持视觉功能。
optimizer substantially outperforms the widely-used AdamW optimizer for LLM training. Kimi K2 was designed to further scale up Moonlight, which employs an architecture similar to DeepSeek-V3. Based on scaling-law analysis, we reduce the number of heads for long-context efficiency, and increase MoE sparsity for greater token efficiency. While scaling up, we encountered a persistent challenge: training instability caused by exploding attention logits, an issue that occurs more frequently with Muo
匹配字段正文
