动态调整猜测树结构，让视觉语言模型推理更快更准。
SAGE: Accelerating Vision-Language Models via Entropy-Guided Adaptive Speculative Decoding
- 根据输出熵实时调整猜测树的深窄或浅宽结构。
- 在不损失质量的前提下，最高提速3.36倍。
- 适合追求高效推理的视觉语言模型应用者。
推测解码已成为加速视觉语言模型（VLMs）推理的有前景方法，通过并行验证多个草稿标记实现。然而，现有方法依赖固定不变的静态树结构，无法适应生成过程中预测难度的变化，导致接受长度不佳且加速有限。本文提出SAGE，一种基于实时预测不确定性的动态推测树结构调整框架。核心思想是输出熵可作为自然置信度指标，具有强时间相关性。SAGE在高置信度时构建更深更窄的树以最大化推测深度，在不确定性高时构建更浅更宽的树以扩大探索范围。该方法提升了接受长度，实现了比静态树基线更快的加速。在多个基准测试中验证了SAGE的有效性：在不损失输出质量的情况下，对LLaVA-OneVision-72B实现最高3.36×的解码加速，对Qwen2.5-VL-72B实现最高3.18×加速。
原文摘要 · Abstract (English)
Speculative decoding has emerged as a promising approach to accelerate inference in vision-language models (VLMs) by enabling parallel verification of multiple draft tokens. However, existing methods rely on static tree structures that remain fixed throughout the decoding process, failing to adapt to the varying prediction difficulty across generation steps. This leads to suboptimal acceptance lengths and limited speedup. In this paper, we propose SAGE, a novel framework that dynamically adjusts the speculation tree structure based on real-time prediction uncertainty. Our key insight is that output entropy serves as a natural confidence indicator with strong temporal correlation across decoding steps. SAGE constructs deeper-narrower trees for high-confidence predictions to maximize speculation depth, and shallower-wider trees for uncertain predictions to diversify exploration. SAGE improves acceptance lengths and achieves faster acceleration compared to static tree baselines. Experiments on multiple benchmarks demonstrate the effectiveness of SAGE: without any loss in output quality, it delivers up to $3.36\times$ decoding speedup for LLaVA-OneVision-72B and $3.18\times$ for Qwen2.5-VL-72B.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。