Thomas Wolf comments on Recursive's new model Flint: quantizing lm_head to 4 bits still comes close to bf16
Hugging Face co-founder Thomas Wolf said that his AGI standard remains AI winning a Nobel Prize, but the AI researchers at Recursive quantized lm_head—which is usually advised to be kept at high precision—to 4 bits, and the result differs from bf16 by only 0.7 percentage points, which impressed him. He also mentioned that the model was released as open weights on Hugging Face, and congratulated Recursive. The quoted original Recursive post introduced that Flint is its first model, positioned as a fast local coding model, built by the automated research system Eureka Machine, reaching 125+ tokens/sec on a MacBook Pro and 150+ on RTX Spark, and built from scratch in under 4 months.
Why read this
AI score 61
The author, using his own AGI standard as a reference, comments on Recursive's new model's approach of quantizing lm_head to 4 bits, and emphasizes that the open weights have been uploaded to Hugging Face.
Topics
View original postIntroducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine. • Best-in-class fast local coding • Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark • Built from scratch in under 4 months Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
View original
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine. • Best-in-class fast local coding • Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark • Built from scratch in under 4 months Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
Body
My bar for AGI is still an AI winning a Nobel.
But an AI researcher looking at the lm_head (that *everyone* tells you to keep in high precision) deciding to quantize it to 4 bits AND being actually right (0.7pt from bf16) is definitely impressive.
Even better when the result are open-weights pushed on the Hugging Gace hub.
Congrats @Recursive_SI !