OpenBMB 转发 MiniCPM-V 4.7 达 370 tokens/sec
370 tokens/sec with MiniCPM-V 4.7?! 🔥
AI 导读
OpenBMB 转发 @Tech2Wild 的测试：MiniCPM v4.7 35B A3B 搭配 DSPARK Drafter，10 秒生成 3,589 tokens，持续吞吐 370.0 tok/s，TTFT 平均 0.30s。OpenBMB 表示很高兴看到社区用投机解码把性能推到极限。
查看原帖So @OpenBMB MiniCPM v4.7 35B A3B Comes With a DSPARK Drafter. This Thing SPITS ! 370 tok/s • Total: 3,589 tokens in 10s • Aggregate: 370.0 tok/s sustained · 359.0 wall-avg · 351.8 peak burst • Per-stream decode: 370.0 high / 370.0 low / 370.0 avg tok/s • TTFT 0.30s avg · E2E 10s avg
正文
当前显示原文，中文全文译文尚未就绪。
370 tokens/sec with MiniCPM-V 4.7?! 🔥
Love seeing the community push the limits of what's possible with speculative decoding.
Huge shoutout to @Tech2Wild for putting this together! 🚀