蚂蚁百灵与英伟达优化 Ling-3.0-flash 推理
In technical collaboration with the @NVIDIAAI team, we optimized inference for the open-weight Ling-3.0-flash model on a single NVIDIA DGX Spark using three te
AI 导读
蚂蚁百灵（Ant Ling）与 NVIDIA AI 团队合作，在单台 NVIDIA DGX Spark 上用三种技术优化了开放权重模型 Ling-3.0-flash 的推理。在 1,319 道 GSM8K 题目上，解码吞吐从 37.81 tok/s 提升至 53.89 tok/s（+42.5%），平均延迟从 15.21s 降至 11.49s（-24.5%），准确率无统计上可检测的变化。
正文
In technical collaboration with the @NVIDIAAI team, we optimized inference for the open-weight Ling-3.0-flash model on a single NVIDIA DGX Spark using three techniques. Across 1,319 GSM8K questions, decode throughput increased 42.5%—from 37.81 to 53.89 tok/s—while mean latency fell 24.5%, from 15.21s to 11.49s, with no statistically detectable accuracy change. 🧵