Vision-language models are slow for the same reason chatbots are: they write one token at a time. Speculative decoding speeds this up by letting a small draft model guess several tokens ahead while the large model checks them in one pass, keeping only the ones it agrees with. MMFlash, a preprint from the National University of Singapore, makes this work better when the model is reading images and video.
The team started from DFlash, a recent drafter that uses block diffusion to propose a whole block of tokens in one forward pass. They measured how much each token depends on the image by comparing the model's predictions with and without the picture. The result was clear: drafted tokens with the least visual dependence were accepted 46% of the time, while the most visual ones were accepted only 22%. The most visual quarter caused 44% of the first rejections in a draft block. Training on multimodal data did not close the gap.
MMFlash's fix is to give extra weight in training to visually dependent tokens that the drafter still predicts badly, easing off as it improves. For long videos, the drafter uses a sparse attention pattern that cuts a context of more than 40,000 tokens to about 1,600. On seven image benchmarks with Qwen3.5 models, average speedups were 3.00 to 3.05 with greedy decoding, 10.3% to 10.9% above DFlash. On three long-video benchmarks, a short fine-tune on 2,000 samples brought speedups of 2.33 to 2.71. It also held up on Gemma4-E4B and Qwen3.8-27B, with speedups up to 4.71 on the larger model.
Because the large model checks every token, the output is identical to normal decoding. That makes this a low-risk optimisation for teams serving open models such as Qwen 3.8 27B or Gemma 4 through SGLang or vLLM, where the code and drafters are released.
The limits show up under load. On SGLang, one Qwen3.5-4B benchmark went from a 3.46 times speedup with one request to 1.61 times with 32 at once, since a busy server already keeps the hardware occupied. The headline 3.58 is a best case. The paper is a preprint marked as work in progress, the results are the authors' own, and video drafting remains harder than images.