AI News

Unsloth releases a local run guide for Qwen3.8-27B and Dynamic V3.1 quantization

Hacker News trendingUpdated Nov 14 12:36

AI summary

Unsloth has released local run documentation for Qwen3.8-27B. The model is a 27B dense model that supports vision and reasoning, has a 256K context window, and its 4-bit quantization can run on 16-19GB of VRAM or on a Mac with 24GB of memory. The documentation also updates the Unsloth Dynamic V3.1 quantization algorithm, claiming that at the same size, accuracy on most benchmarks improves by 5-10%, covering 2-bit to 6-bit quantization, with 3-bit unchanged. The article also gives deployment methods for llama.cpp, Unsloth Desktop, and vLLM, as well as NVFP4 quantization running at about 1.5 times the speed of BF16 on Blackwell GPUs.

Why read this

AI score 64

Unsloth has provided a local quantized run solution and hardware requirements for Qwen3.8-27B, so you can use it to judge whether your own device can run it.

Topics

Tags

Body

Only a summary is authorized here. Continue at the validated source link.

Source

Hacker News trending