Thinking Machines Lab 开源 594B 参数模型 Inkling 2 Small
Thinking Machines Lab kept its promise and open-sourced Inkling 2 Small, a 594B-parameter model trained with 4-bit forward passes.
Thinking Machines Lab 开源了 594B 参数的 Inkling 2 Small,已在 HuggingFace 以 Apache 2.0 许可提供下载。该模型从预训练第一步到最终 RL rollout 全程采用原生 4-bit 前向计算,预训练时间缩短 8%、RL 时间缩短 20%,效率损失约 2%。运行至少需要 500GB GPU 显存(NVFP4)或 800GB(MXFP8),MXFP8 在评测上仅比 NVFP4 高 1–2 个百分点;由于训练全程保持相同数值格式,发布的 4-bit checkpoint 无需单独校准。
推荐理由
AI 评分 71
原文给出了 4-bit 前向训练带来的时间节省与精度损失,读者可据此判断低比特训练的实际取舍。
主题
查看原帖用更少的比特进行训练速度更快,但量化误差可能会干扰学习。 今天在 Connectionism 上,我们详细介绍了如何从预训练的第一步一直到最终的 RL rollout,全程以原生 4-bit 前向计算来训练 Inkling 2 Small。 https://t.co/CIMECSEpD0
查看原文
Training with fewer bits is faster, but quantization error can interfere with learning. Today on Connectionism, we detail how we trained Inkling 2 Small with a native 4-bit forward pass from the first step of pretraining through the final RL rollout. https://t.co/CIMECSEpD0
正文
Thinking Machines Lab kept its promise and open-sourced Inkling 2 Small, a 594B-parameter model trained with 4-bit forward passes. > now available for download on HuggingFace under an Apache 2.0 licens > Training in 4-bit cut pretraining time 8% and RL time 20%, for about 2% efficiency loss.
> Running it takes at least 500GB of GPU memory in NVFP4 and at least 800GB in MXFP8, and MXFP8 gains “a very marginal 1–2 percentage points” over NVFP4 on evals.
> Because training kept the same numerics from pretraining through the final RL rollout, the released 4-bit checkpoint needed no separate calibration step.
> The lab first tried NVFP4 in both forward and backward passes, but that hurt compute efficiency too much, so only the forward pass stays in 4-bit.