Liquid AI 发布 LFM2.5-VL-DSpark，为视觉语言模型加速

How does speculative decoding work for VLMs

The vision drafter uses the same architecture as our text LFM2.5-DSpark drafters: it captures the target model's hidden states at a fixed set of tapped layers and conditions on them to draft a block of k candidate tokens. Image patches and text tokens are projected into a shared representation before those layers, so the drafter operates on hidden-state vectors of identical dimensionality regardless of input modality. The inference algorithm is therefore unchanged from the text models.

Training and Architecture

We follow the DSpark recipe with a mixture of vision-language SFT data, weighted toward the workloads we expect the model to serve. Based on ablations across 3, 4, and 5 layers, the draft model is a simplified attention-only drafter with 4 layers and a block size of 9. We ran 10 epochs on the final mixture and measured acceptance after each, which improved with additional training tokens before reaching diminishing returns. At inference time, we recommend a block size of 8 or 9 depending on the hardware.

The resulting drafter has approximately 280M parameters and increases the deployed model’s parameter count by just 8.9%.

Inference Speedup on CPU and GPU

The DSpark draft model for LFM2.5-VL-3B ships with day-one support for llama.cpp , MLX-VLM , and SGLang .

We measure both on-device inference and GPU inference. Both configurations use a DSpark block size of 8 and are evaluated on six diverse vision-based tasks, including general VQA, text VQA, image captioning, chart VQA, complex reasoning, and multi-turn conversation, following the MMSpec benchmark .

On-device inference. With MLX on an M5 Max, decoding runs 2.30x to 3.13x faster by task. End-to-end latency improves by 1.56x to 2.62x. With llama.cpp on an M3 Ultra, decoding improves by 1.57x to 2.14x and end-to-end by 1.30x to 1.77x.

GPU inference. On H100, the same drafter delivers 20.4x to 2.66x faster decoding, with end-to-end improvements of 1.64x to 2.27x.

Limitations of speculation for vision workloads

In LLMs, prefill is mostly compute-bound, and its cost grows (sub)quadratically with prompt length. VLMs add to this because the image first passes through a vision encoder, then the language backbone processes hundreds of visual tokens along with the text prompt. Edge devices have far less compute than datacenter GPUs, so prefill takes up more of the end-to-end latency, as time-to-first-token and decode measurements on Apple silicon and H100 show. (The M5's per-core GPU neural accelerators narrow this gap).

Speculative decoding speeds up only decode, not vision encoding or prefill. When those stages already take up much of the wall time, even a large decode speedup gives only a modest end-to-end gain. This is Amdahl's law, where the overall speedup is capped by the part of the workload that isn't accelerated.

How to use LFM2.5-VL-DSpark

Running the DSpark draft models with SGLang requires an SGLang build with DSpark support for LFM2 targets ( PR #40651 ). Launch the target with the draft attached:

python -m sglang.launch_server \ --model-path LiquidAI/LFM2.5-VL-3B \ --speculative-algorithm DSPARK \ --speculative-draft-model-path LiquidAI/LFM2.5-VL-3B-DSpark \ --speculative-draft-attention-backend flashinfer \ --speculative-dspark-block-size 9 \ --disable-radix-cache

Then query the OpenAI-compatible endpoint at http://localhost:30000/v1 . The block size is read from the draft's config.json ; the baseline is the same command without the three --speculative-* flags.

Running them with llama.cpp requires the respective llama.cpp build ( PR#29339 ).

llama-server -m models/LFM2.5-VL-3B-F16.gguf \ --mmproj models/mmproj-LFM2.5-VL-3B-F16.gguf \ -md LFM2.5-2.6B-DSpark-F16.gguf \ --spec-type draft-dspark --spec-draft-n-max 8 --spec-draft-n-min 0 \ -fa on -ngl 99 -c 8192

Running them with MLX-VLM requires the respective build ( PR#2280 ).

mlx_vlm.server --model LiquidAI/LFM2.5-VL-3B --draft-model LiquidAI/LFM2.5-VL-3B-DSpark

The block size is read from the sidecar metadata (n-max is clamped to it). Speculative decoding is exact : the target verifies every proposed token, so greedy output equals the target alone; per-response timings report draft_n / draft_n_accepted .

Our vision DSpark draft model is available on Hugging Face in Safetensors and GGUF formats .

With LFM2.5, we're delivering on our vision of AI that runs anywhere. These models are:

Open-weight — Download, fine-tune, and deploy without restrictions.

Fast from day one — Day-one support for llama.cpp, MLX, and SGLang.

A complete family — From base models for customization to specialized audio and vision variants, one architecture covers diverse use cases

We can’t wait to see what you build.

Citation

For citations, please use the following reference or BibTeX: Liquid AI, "LFM2.5-VL-DSpark: Accelerating vision-language models on edge and beyond", Liquid AI Blog, Sep 2026.

@article{liquidAI2026vldspark, author = {Liquid AI}, title = {LFM2.5-VL-DSpark: Accelerating vision-language models on edge and beyond}, journal = {Liquid AI Blog}, year = {2026}, note = {www.liquid.ai/blog/lfm2-5-vl-dspark}, }

Models mentioned in this article 3 LiquidAI/LFM2.5-VL-3B Image-Text-to-Text • 3B • Updated 7 days ago • 26.6k • 214 LiquidAI/LFM2.5-VL-3B-DSpark Image-Text-to-Text • 0.3B • Updated 7 days ago • 656 • 17 LiquidAI/LFM2.5-VL-3B-DSpark-GGUF Image-Text-to-Text • 0.3B • Updated 7 days ago • 1.06k • 9 Papers mentioned in this article 2 MMSpec: Benchmarking Speculative Decoding for Vision-Language Models Paper • 2603.14989 • Published Mar 16 DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation Paper • 2607.05147 • Published Jul 6 • 50 More from this author

Up to 3.2x Faster Inference with LFM2.5-DSpark

56 August 20, 2026 LFM2.5 Q4\_0 Checkpoints from Quantization-Aware Distillation 36 August 19, 2026 Community how this project works in the real world \n","updatedAt":"2026-09-26T04:17:31.554Z","author":{"_id":"6ab745de219365e1d28b20b9","avatarUrl":"/avatars/420c4aca516f2b48940c86204cf3042d.svg","fullname":"Shreyash Rampurkar","name":"shreyash2512","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.775952160358429},"editors":["shreyash2512"],"editorAvatarUrls":["/avatars/420c4aca516f2b48940c86204cf3042d.svg"],"reactions":[],"isReport":false}},{"id":"6ab7b14e41e973b84d6e720e","author":{"_id":"6a3cd0fb3af2b0b0921e329a","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/1Yp81w2J8OBsuzrHcQoZu.jpeg","fullname":"Elton Williams","name":"eltonwilliams","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false},"createdAt":"2026-09-26T11:49:34.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":">The vision drafter uses the same architecture as our text LFM2.5-DSpark drafters: it captures the target model's hidden states at a fixed set of tapped layers and conditions on them to moviesda draft a block of k candidate tokens. Image patches and text tokens are projected into a shared representation before those layers, so the drafter operates on hidden-state vectors of identical dimensionality regardless of input modality. The inference algorithm is therefore unchanged from the text models.\n\nThis is a really interesting step toward making vision-language models faster and more practical in real-world applications. The reported speedups of up to 3.13x on-device and 2.66x on an H100 are especially notable because the approach aims to improve inference speed without changing the output quality, while keeping the additional memory requirement relatively small.\n\nI also like how the DSpark approach carries the same core idea from text models into the vision-language setting. Having the image patches and text tokens projected into a shared representation means the drafter can work with the same hidden-state dimensions across both modalities, which makes the speculative decoding process much more straightforward and reusable.\n\nThe day-one support for llama.cpp, MLX-VLM, and SGLang also makes this especially interesting for developers experimenting with local and accelerated VLM inference. It will be worth watching how these gains translate across different hardware and workloads as the experimental model develops.","html":" \n The vision drafter uses the same architecture as our text LFM2.5-DSpark drafters: it captures the target model's hidden states at a fixed set of tapped layers and conditions on them to moviesda draft a block of k candidate tokens. Image patches and text tokens are projected into a shared representation before those layers, so the drafter operates on hidden-state vectors of identical dimensionality regardless of input modality. The inference algorithm is therefore unchanged from the text models. \n \n This is a really interesting step toward making vision-language models faster and more practical in real-world applications. The reported speedups of up to 3.13x on-device and 2.66x on an H100 are especially notable because the approach aims to improve inference speed without changing the output quality, while keeping the additional memory requirement relatively small. \n I also like how the DSpark approach carries the same core idea from text models into the vision-language setting. Having the image patches and text tokens projected into a shared representation means the drafter can work with the same hidden-state dimensions across both modalities, which makes the speculative decoding process much more straightforward and reusable. \n The day-one support for llama.cpp, MLX-VLM, and SGLang also makes this especially interesting for developers experimenting with local and accelerated VLM inference. It will be worth watching how these gains translate across different hardware and workloads as the experimental model develops. \n","updatedAt":"2026-09-26T11:49:34.621Z","author":{"_id":"6a3cd0fb3af2b0b0921e329a","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/1Yp81w2J8OBsuzrHcQoZu.jpeg","fullname":"Elton Williams","name":"eltonwilliams","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9159632325172424},"editors":["eltonwilliams"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/1Yp81w2J8OBsuzrHcQoZu.jpeg"],"reactions":[],"isReport":false}},{"id":"6ab9ff8bd2a8d6c1953f6e89","author":{"_id":"6978331343d82bbc94777e09","avatarUrl":"/avatars/a8de22408ceb186b738fa752bb13e0f9.svg","fullname":"hamdy hosny","name":"mady4host","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2026-09-28T05:47:55.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":" watch the matches SiirTv \n watch the matches Yallashoot \n watch the matches Yallashootkora ","html":" watch the matches SiirTv watch the matches Yallashoot watch the matches Yallashootkora \n","updatedAt":"2026-09-28T05:47:55.222Z","author":{"_id":"6978331343d82bbc94777e09","avatarUrl":"/avatars/a8de22408ceb186b738fa752bb13e0f9.svg","fullname":"hamdy hosny","name":"mady4host","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.40826740860939026},"editors":["mady4host"],"editorAvatarUrls":["/avatars/a8de22408ceb186b738fa752bb13e0f9.svg"],"reactions":[],"isReport":false}},{"id":"6abc66ada4f7d0597146d2a4","author":{"_id":"6abc6589ba510dd2287c1d92","avatarUrl":"/avatars/ff3d70b744c968d01110a1b9f17921f7.svg","fullname":"Daniel Salameh","name":"danielsalameh","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2026-09-30T01:32:29.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"U","html":" U \n","updatedAt":"2026-09-30T01:32:29.639Z","author":{"_id":"6abc6589ba510dd2287c1d92","avatarUrl":"/avatars/ff3d70b744c968d01110a1b9f17921f7.svg","fullname":"Daniel Salameh","name":"danielsalameh","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"ja","probability":0.6873694658279419},"editors":["danielsalameh"],"editorAvatarUrls":["/avatars/ff3d70b744c968d01110a1b9f17921f7.svg"],"reactions":[],"isReport":false}}],"status":"open","isReport":false,"pinned":false,"locked":false,"collection":"community_blogs"},"contextAuthors":["tugot17","ykhrustalev","iamleonie","vivianamarquez"],"primaryEmailConfirmed":false,"discussionRole":0,"acceptLanguages":["en","zh"],"withThread":true,"cardDisplay":false,"repoDiscussionsLocked":false,"hideComments":true}"> WattoCyber 6 days ago • This comment has been hidden (marked as Spam) shreyash2512 5 days ago how this project works in the real world

Reply eltonwilliams 5 days ago The vision drafter uses the same architecture as our text LFM2.5-DSpark drafters: it captures the target model's hidden states at a fixed set of tapped layers and conditions on them to moviesda draft a block of k candidate tokens. Image patches and text tokens are projected into a shared representation before those layers, so the drafter operates on hidden-state vectors of identical dimensionality regardless of input modality. The inference algorithm is therefore unchanged from the text models.

This is a really interesting step toward making vision-language models faster and more practical in real-world applications. The reported speedups of up to 3.13x on-device and 2.66x on an H100 are especially notable because the approach aims to improve inference speed without changing the output quality, while keeping the additional memory requirement relatively small.

I also like how the DSpark approach carries the same core idea from text models into the vision-language setting. Having the image patches and text tokens projected into a shared representation means the drafter can work with the same hidden-state dimensions across both modalities, which makes the speculative decoding process much more straightforward and reusable.

The day-one support for llama.cpp, MLX-VLM, and SGLang also makes this especially interesting for developers experimenting with local and accelerated VLM inference. It will be worth watching how these gains translate across different hardware and workloads as the experimental model develops.

Reply mady4host 3 days ago watch the matches SiirTv watch the matches Yallashoot watch the matches Yallashootkora

Reply danielsalameh 1 day ago U

Reply Edit Preview Upload images, audio, and videos by dragging in the text input, pasting, or clicking here . Tap or paste here to upload images Comment · Sign up or log in to comment

+37

Models mentioned in this article 3

LiquidAI/LFM2.5-VL-3B

LiquidAI/LFM2.5-VL-3B-DSpark

LiquidAI/LFM2.5-VL-3B-DSpark-GGUF

Papers mentioned in this article 2