Understand this story
In ComfyUI v0.45.0, the text generation hot path for the Qwen3.x series of LLMs was rewritten, increasing token generation speed for the same checkpoint on the same GPU by up to two times, with no change in model or output quality. According to information published on the Comfy Blog, on an RTX 5090, Qwen3.5 9B improved from 106 to 219 tokens/s, and on an RTX 5080 it improved from 91 to 129 tokens/s; for Qwen3.8 27B describing a 1536×1536 image, on an RTX 5090 it improved from 55 to 122 tokens/s. The improvements include a GEMV kernel that rearranges weights to address the VRAM read bottleneck, adaptive tree MTP speculative decoding, and the newly added Comfy-Kitchen attention (which can be enabled with --use-ck-attention), effective on both NVIDIA and AMD, and on both Windows and Linux. There are currently no further developments.
AI-generated from public reports
Latest development · Report
ComfyUI v0.45.0 Speeds Up Qwen3.x Local Text Generation by Up to 2xDevelopments
1 confirmed developments
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 12
Comfy BlogFirst-partySelected
ComfyUI v0.45.0 Speeds Up Qwen3.x Local Text Generation by Up to 2x
In v0.45.0, ComfyUI rewrote the text generation hot path for the Qwen3.x series of LLMs, increasing token generation speed by up to 2x for the same checkpoint on the same GPU, with no change to the model or output quality. On an RTX 5090, Qwen3.5 9B improved from 106 to 219 tokens/s, and on an RTX 5080 from 91 to 129 tokens/s; Qwen3.8 27B describing a 1536×1536 image improved from 55 to 122 tokens/s on an RTX 5090. The improvements include a GEMV kernel that reorders weights to address the VRAM read bottleneck, adaptive tree MTP speculative decoding, and newly added Comfy-Kitchen attention (can be enabled with --use-ck-attention), effective on both NVIDIA and AMD, Windows and Linux.