Unsloth releases the Dynamic GGUF quantized version of Flint, runnable locally on 24GB of memory
Unsloth has released the Dynamic GGUF quantized version of Flint, which can run locally on 24GB-memory devices, along with a running guide. Flint is Recursive's Apache-2.0 model with 39.8B parameters and 2.5B active, which Unsloth calls the strongest MoE model under 50B parameters, reaching 125 t/s on Mac and 150 t/s on RTX Spark. Recursive co-founder Richard Socher previously introduced that the model was built from scratch in under 4 months by the automated research system Eureka Machine, and the weights are now open on Hugging Face.
Why read this
SelectedAI score 63
Unsloth provides the GGUF quantized version of Flint along with a local running guide, so readers can use it to actually deploy this 39.8B-parameter MoE model on a 24GB-memory device.
Topics
View original postIntroducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine. • Best-in-class fast local coding • Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark • Built from scratch in under 4 months Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
View original
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine. • Best-in-class fast local coding • Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark • Built from scratch in under 4 months Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
Body
Run Flint locally on 24GB RAM via Unsloth Dynamic GGUFs!
Flint is the strongest MoE model under 50B parameters.
Recursive’s Apache-2.0 model has 39.8B parameters (2.5B active). Run at 125 t/s on Mac, 150 on RTX Spark