TIGER提升多模态生成速度,让轻量模型更准地预测视觉关键内容。
TIGER: Text-Conditioned Visual Gated Routing with Acceptance Alignment for Multimodal Speculative Decoding

- 根据文本状态动态筛选视觉关键片段,避免冗余信息干扰
- 通过验证器反馈优化预测长度,使推测结果更长且更易通过验证
- 适合需要高速多模态生成的场景,如实时视频描述或图像生成
推测解码通过轻量级起草模型提出多个词元,由大型目标模型验证以加速自回归生成。尽管在纯文本大模型中有效,但现有方法在视觉语言模型(VLM)中效果有限,因起草模型常在视觉关键内容上偏离。现有方法未直接处理无关视觉证据,也未优化验证器接受前缀长度这一影响速度的核心因素。本文提出TIGER:一种基于文本条件的视觉门控路由框架,动态根据起草模型当前文本状态选择稀疏的上下文相关视觉词元,而非暴露完整视觉词元集或固定压缩接口。为更好对齐训练与推理效率,采用基于验证器反馈奖励的接受对齐组策略训练,以接受前缀长度为目标优化起草模型,结合知识蒸馏预热与KL锚定。这促使起草模型不仅模仿目标模型,还生成更长可接受的推测续写。实验表明,TIGER在精确验证侧推测解码下持续提升接受前缀长度与推测加速比,在视觉路由分析中实现质量-延迟良好权衡,下游准确率与基线相当。
原文摘要 · Abstract (English)
Speculative decoding accelerates autoregressive generation by letting a lightweight drafter propose multiple tokens that are verified by a larger target model. Although effective for text-only LLMs, speculative decoding yields limited gains in VLMs because drafters often diverge on vision-critical content, while existing multimodal acceleration methods do not directly address irrelevant visual evidence or optimize the verifier-accepted prefix length that governs speedup. We propose TIGER, a Text-conditioned vIsual GatEd Routing framework for multimodal speculative decoding. TIGER dynamically selects a sparse set of context-relevant visual tokens based on the drafter's current textual state, rather than expose the full visual token set or a fixed compressed interface. To better align training with inference-time efficiency, we optimize the drafter with acceptance-aligned group-based policy training using verifier-derived rewards based on accepted prefix length, built on top of distillation warm start with KL anchoring. This encourages the drafter not only to imitate the target model, but also to produce speculative continuations that survive verification for longer prefixes. Experiments show that TIGER yields consistent gains in accepted prefix length and speculative speedup under exact verifier-side speculative decoding, while achieving favorable quality-latency trade-offs with comparable downstream accuracy in visual-routing analyses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。