🛠️Liquid AI's VLM Drafter Speeds Up 3B Vision Decoding by 3.1x
TL;DR
Liquid AI released LFM2.5-VL-DSpark, a 280M-parameter speculative decoding drafter for its 3B vision-language model. It speeds decoding up to 3.13x on an M5 Max and 2.66x on H100 with identical output.
Liquid AI released LFM2.5-VL-DSpark, a 280M-parameter speculative decoding drafter for its 3B vision-language model. It speeds decoding up to 3.13x on an M5 Max and 2.66x on H100 with identical output.
Key Points
The drafter adds 279.5M parameters, 8.9% on top of LFM2.5-VL-3B, as a 4-layer attention-only model with block size 9
MLX on an M5 Max: 2.30x to 3.13x faster decode, 1.56x to 2.62x end to end
H100: up to 2.66x faster decode and 1.64x to 2.27x end to end
Day-one support in llama.cpp, MLX-VLM and SGLang; weights ship in Safetensors and GGUF
Amdahl's law caveat: vision encoding and prefill are untouched, so edge gains shrink on image-heavy prompts
Why It Matters
It's a clear worked example of where speculative decoding pays off for multimodal models and where it stops. Anyone shipping edge VLMs can drop it in today with three CLI flags.
Quick Facts
Comments
Be the first to comment
Enjoyed this article?
Get it daily. 7am. Free. Reads in 5 minutes.
Join 3,502 builders reading daily.