AI News

Thinking Machines Lab 开源 594B 参数模型 Inkling 2 Small

Rohan Paul更新于 11月14日 08:15

Thinking Machines Lab kept its promise and open-sourced Inkling 2 Small, a 594B-parameter model trained with 4-bit forward passes.

AI 导读

Thinking Machines Lab 开源了 594B 参数的 Inkling 2 Small,已在 HuggingFace 以 Apache 2.0 许可提供下载。该模型从预训练第一步到最终 RL rollout 全程采用原生 4-bit 前向计算,预训练时间缩短 8%、RL 时间缩短 20%,效率损失约 2%。运行至少需要 500GB GPU 显存(NVFP4)或 800GB(MXFP8),MXFP8 在评测上仅比 NVFP4 高 1–2 个百分点;由于训练全程保持相同数值格式,发布的 4-bit checkpoint 无需单独校准。

推荐理由

AI 评分 71

原文给出了 4-bit 前向训练带来的时间节省与精度损失,读者可据此判断低比特训练的实际取舍。

主题

标签

引用

Thinking Machines

@thinkymachines

用更少的比特进行训练速度更快,但量化误差可能会干扰学习。 今天在 Connectionism 上,我们详细介绍了如何从预训练的第一步一直到最终的 RL rollout,全程以原生 4-bit 前向计算来训练 Inkling 2 Small。 https://t.co/CIMECSEpD0

查看原文

Training with fewer bits is faster, but quantization error can interfere with learning. Today on Connectionism, we detail how we trained Inkling 2 Small with a native 4-bit forward pass from the first step of pretraining through the final RL rollout. https://t.co/CIMECSEpD0

查看原帖

正文

Thinking Machines Lab kept its promise and open-sourced Inkling 2 Small, a 594B-parameter model trained with 4-bit forward passes. > now available for download on HuggingFace under an Apache 2.0 licens > Training in 4-bit cut pretraining time 8% and RL time 20%, for about 2% efficiency loss.

> Running it takes at least 500GB of GPU memory in NVFP4 and at least 800GB in MXFP8, and MXFP8 gains “a very marginal 1–2 percentage points” over NVFP4 on evals.

> Because training kept the same numerics from pretraining through the final RL rollout, the released 4-bit checkpoint needed no separate calibration step.

> The lab first tried NVFP4 in both forward and backward passes, but that hurt compute efficiency too much, so only the forward pass stays in 4-bit.

来源

Rohan Paul