让视频大模型生成更快，靠视觉关键词精准定位+宽松匹配
See the Forest for the Trees: Loosely Speculative Decoding via Visual-Semantic Guidance for Efficient Inference of Video LLMs
- 用视觉关键词定位关键生成片段，其他内容可宽松匹配
- 在保持99.8%性能下，提速2.7倍至2.94倍
- 适合追求高效推理的视频大模型研究与应用者
视频大语言模型（Video-LLMs）在视频理解上表现优异，但自回归生成时推理延迟高。推测解码（SD）通过先生成草案再验证的方式缓解该问题，但现有方法受限于严格的精确匹配规则，严重制约加速潜力。为此，我们提出首个无需训练的松散推测解码框架LVSpec，针对视频生成中稀疏的视觉相关锚点（需严格匹配）与大量无关填充内容（可宽松验证）的特点，设计轻量级视觉相关标记识别机制以精准定位前者，并引入位置偏移容错机制，有效挽救语义等价但位置不符的标记。实验表明，LVSpec在保持>99.8%目标性能的同时，使Qwen2.5-VL-32B加速2.70倍，LLaVA-OneVision-72B加速2.94倍，相比当前最优无训练SD方法，平均接受长度和加速比分别提升136%和35%。
原文摘要 · Abstract (English)
Video Large Language Models (Video-LLMs) excel in video understanding but suffer from high inference latency during autoregressive generation. Speculative Decoding (SD) mitigates this by applying a draft-and-verify paradigm, yet existing methods are constrained by rigid exact-match rules, severely limiting the acceleration potential. To bridge this gap, we propose LVSpec, the first training-free loosely SD framework tailored for Video-LLMs. Grounded in the insight that generation is governed by sparse visual-relevant anchors (mandating strictness) amidst abundant visual-irrelevant fillers (permitting loose verification), LVSpec employs a lightweight visual-relevant token identification scheme to accurately pinpoint the former. To further maximize acceptance, we augment this with a position-shift tolerant mechanism that effectively salvages positionally mismatched but semantically equivalent tokens. Experiments demonstrate that LVSpec achieves high fidelity and speed: it preserves >99.8 of target performance while accelerating Qwen2.5-VL-32B by 2.70x and LLaVA-OneVision-72B by 2.94x. Notably, it boosts the mean accepted length and speedup ratio by 136% and 35% compared to SOTA training-free SD methods for Video-LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。