AI News

Trending story

ComfyUI v0.45.0 rewrites the LLM hot path, boosting token generation by up to two times

1 reports1 story updatesUpdated Nov 12 00:56

Understand this story

AI overview

In ComfyUI v0.45.0, the text generation hot path for the Qwen3.x series of LLMs was rewritten, increasing token generation speed for the same checkpoint on the same GPU by up to two times, with no change in model or output quality. According to information published on the Comfy Blog, on an RTX 5090, Qwen3.5 9B improved from 106 to 219 tokens/s, and on an RTX 5080 it improved from 91 to 129 tokens/s; for Qwen3.8 27B describing a 1536×1536 image, on an RTX 5090 it improved from 55 to 122 tokens/s. The improvements include a GEMV kernel that rearranges weights to address the VRAM read bottleneck, adaptive tree MTP speculative decoding, and the newly added Comfy-Kitchen attention (which can be enabled with --use-ck-attention), effective on both NVIDIA and AMD, and on both Windows and Linux. There are currently no further developments.

AI-generated from public reports

Latest development · Report

ComfyUI v0.45.0 Speeds Up Qwen3.x Local Text Generation by Up to 2x

Developments

1 confirmed developments

  1. ComfyUI rewrites the LLM text generation hot path to improve token rate

    Source:Comfy Blog

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 12

  1. Comfy BlogFirst-partySelected

    ComfyUI v0.45.0 Speeds Up Qwen3.x Local Text Generation by Up to 2x

    In v0.45.0, ComfyUI rewrote the text generation hot path for the Qwen3.x series of LLMs, increasing token generation speed by up to 2x for the same checkpoint on the same GPU, with no change to the model or output quality. On an RTX 5090, Qwen3.5 9B improved from 106 to 219 tokens/s, and on an RTX 5080 from 91 to 129 tokens/s; Qwen3.8 27B describing a 1536×1536 image improved from 55 to 122 tokens/s on an RTX 5090. The improvements include a GEMV kernel that reorders weights to address the VRAM read bottleneck, adaptive tree MTP speculative decoding, and newly added Comfy-Kitchen attention (can be enabled with --use-ck-attention), effective on both NVIDIA and AMD, Windows and Linux.

Related stories

  1. Related storiesAnthropic launches Claude Fable 5.2 and Claude Mythos 5.2
  2. Related storiesBaidu Wenxin releases ERNIE 5.5 Flash Preview
  3. Related storiesGoogle launches Gemini 4 Hydrogen on Antigravity
  4. Related storiesUnsloth says Qwen3.8-Flash-Next GGUF runs faster