ModelBest responds to MiniCPM5 small model benchmark comparison
GooGZ AI tested three small models under 2B on an RTX 3080 with PA Core, saying Spark X2.6 1.7B, LFM3 and MiniCPM5 each have their strengths, with MiniCPM5 performing the most balanced. ModelBest OpenBMB reposted the test and thanked them, saying they are delighted to see how far small models have progressed.
View original postSmall models are going through a real growth spurt right now. On my personal-agent tests (PA Core, reasoning on, RTX 3080), three models at 2B and under now land within 21 points of each other, and rubbing shoulders with older 4B/9B weights. Newest on the bench: Spark X2.6 1.7B from @SparkLLM - PA total: 172.3/300 - Tool calls: 30/30 in every run - Context recall: 9.3 at 8K, 8.7 at 16K, best of the three! - Schedule and draft (PA2): 98 and 100 in two runs, one 54 pulled the mean to 84 - Daily organization (PA1): 35.7, the weak spot Under 2B, three different strengths. @liquidai LFM3 plans, @SparkLLM Spark reads, @OpenBMB MiniCPM5 is the balanced one. Leave thinking on. With it off, Spark is a much weaker model on these tasks.
View original
Small models are going through a real growth spurt right now. On my personal-agent tests (PA Core, reasoning on, RTX 3080), three models at 2B and under now land within 21 points of each other, and rubbing shoulders with older 4B/9B weights. Newest on the bench: Spark X2.6 1.7B from @SparkLLM - PA total: 172.3/300 - Tool calls: 30/30 in every run - Context recall: 9.3 at 8K, 8.7 at 16K, best of the three! - Schedule and draft (PA2): 98 and 100 in two runs, one 54 pulled the mean to 84 - Daily organization (PA1): 35.7, the weak spot Under 2B, three different strengths. @liquidai LFM3 plans, @SparkLLM Spark reads, @OpenBMB MiniCPM5 is the balanced one. Leave thinking on. With it off, Spark is a much weaker model on these tasks.
Body
Love seeing these real-world comparisons! 💙 Thanks for including MiniCPM5 in your tests. Exciting to see how far small models have come! 🚀