AI News

Trending story

Unsloth releases Qwen3.8-27B GGUF quantization using Dynamic V3.1

1 reports1 story updatesUpdated Nov 12 01:06

Understand this story

AI overview

Unsloth has released local-run documentation for Qwen3.8-27B and launched GGUF quantized versions using Unsloth Dynamic V3.1. According to this document, which was featured on Hacker News, Qwen3.8-27B is a 27B dense model that supports vision and reasoning, has a 256K context window, and its 4-bit quantization can run on a Mac with 16-19GB of VRAM or 24GB of RAM. The document also updates the Unsloth Dynamic V3.1 quantization algorithm, claiming that at the same size, accuracy on most benchmarks improves by 5-10%, covering 2-bit to 6-bit quantization, while 3-bit remains unchanged. The article also provides deployment methods for llama.cpp, Unsloth Desktop, and vLLM, as well as the roughly 1.5x speed of NVFP4 quantization over BF16 on Blackwell GPUs. There are currently no subsequent developments.

AI-generated from public reports

Latest development · Report

Unsloth releases a local run guide for Qwen3.8-27B and Dynamic V3.1 quantization

Developments

1 confirmed developments

  1. Unsloth releases Qwen3.8-27B GGUF quantization using Dynamic V3.1

    Source:English text is being prepared.

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 12

  1. Hacker News trending

    Unsloth releases a local run guide for Qwen3.8-27B and Dynamic V3.1 quantization

    Unsloth has released local run documentation for Qwen3.8-27B. The model is a 27B dense model that supports vision and reasoning, has a 256K context window, and its 4-bit quantization can run on 16-19GB of VRAM or on a Mac with 24GB of memory. The documentation also updates the Unsloth Dynamic V3.1 quantization algorithm, claiming that at the same size, accuracy on most benchmarks improves by 5-10%, covering 2-bit to 6-bit quantization, with 3-bit unchanged. The article also gives deployment methods for llama.cpp, Unsloth Desktop, and vLLM, as well as NVFP4 quantization running at about 1.5 times the speed of BF16 on Blackwell GPUs.

Related stories

  1. Related storiesDeepSeek releases DeepSeek-V4.1-Flash-1017
  2. Related storiesEnglish translation is being prepared.
  3. Related storiesGeorgi Gerganov announces the release of llama.cpp v1.0.0
  4. Related storiesMiniMax releases the H3-Image image model and opens the weights