用动态树结构推测解码，让视觉语言模型提速三倍还不丢质量。
Spec-LLaVA: Accelerating Vision-Language Models with Dynamic Tree-Based Speculative Decoding
- 用轻量模型预猜后续文本，大模型并行验证，一步生成多词
- 在MS COCO上对7B/13B模型实现最高3.28倍加速，质量无损
- 适合部署在设备端的低资源环境，助力实时多模态助手
视觉语言模型（VLMs）具备强大的多模态推理能力，但自回归推理速度慢，限制了其在实时应用中的部署。本文提出Spec-LLaVA，通过推测解码加速VLM，且不牺牲生成质量。该系统将一个轻量级草稿VLM与大型目标模型配对：草稿模型推测未来词元，目标模型并行验证，实现每步生成多个词元。为提升效率，设计动态树状验证算法，根据草稿模型置信度自适应扩展与剪枝推测分支。在MS COCO外域图像上，Spec-LLaVA对LLaVA-1.5（7B, 13B）模型实现最高3.28×加速，生成质量无损失。本工作提出一种无损加速框架，采用动态树结构推测解码，为实用化实时多模态助手开辟路径。轻量级草稿模型设计使其适用于资源受限或设备端部署场景。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) enable powerful multimodal reasoning but suffer from slow autoregressive inference, limiting their deployment in real-time applications. We introduce Spec-LLaVA, a system that applies speculative decoding to accelerate VLMs without sacrificing output quality. Spec-LLaVA pairs a lightweight draft VLM with a large target model: the draft speculates future tokens, which the target verifies in parallel, allowing multiple tokens to be generated per step. To maximize efficiency, we design a dynamic tree-based verification algorithm that adaptively expands and prunes speculative branches using draft model confidence. On MS COCO out-of-domain images, Spec-LLaVA achieves up to 3.28$\times$ faster decoding on LLaVA-1.5 (7B, 13B) with no loss in generation quality. This work presents a lossless acceleration framework for VLMs using dynamic tree-structured speculative decoding, opening a path toward practical real-time multimodal assistants. Importantly, the lightweight draft model design makes the framework amenable to resource-constrained or on-device deployment settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。