提出V-Skip方法，让多模态推理更高效且不丢关键视觉信息。
Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoring
- 用双路径门控机制结合语言和跨模态注意力评估令牌重要性
- 实现2.9倍加速，文档问答任务性能提升超30%
- 适合需要快速准确多模态推理的应用场景
尽管思维链（CoT）推理显著提升了多模态大模型（MLLMs）的性能，但其自回归特性带来了巨大的延迟瓶颈。现有通过标记压缩缓解该问题的方法常盲目套用以文本为中心的度量标准，导致在多模态上下文中出现严重错误。我们识别出一种关键失效模式——视觉遗忘，即语言上冗余的标记被错误删除，引发幻觉。为此，我们提出V-Skip，将标记压缩重新建模为视觉锚定的信息瓶颈（VA-IB）优化问题。V-Skip采用双路径门控机制，通过语言意外度与跨模态注意力流共同衡量标记重要性，有效保留视觉显著锚点。在Qwen2-VL和Llama-3.2系列上的大量实验表明，V-Skip实现2.9倍加速，精度损失可忽略；尤其在细粒度视觉细节保留方面表现优异，在DocVQA任务上优于其他基线超过30%。
原文摘要 · Abstract (English)
While Chain-of-Thought (CoT) reasoning significantly enhances the performance of Multimodal Large Language Models (MLLMs), its autoregressive nature incurs prohibitive latency constraints. Current efforts to mitigate this via token compression often fail by blindly applying text-centric metrics to multimodal contexts. We identify a critical failure mode termed Visual Amnesia, where linguistically redundant tokens are erroneously pruned, leading to hallucinations. To address this, we introduce V-Skip that reformulates token pruning as a Visual-Anchored Information Bottleneck (VA-IB) optimization problem. V-Skip employs a dual-path gating mechanism that weighs token importance through both linguistic surprisal and cross-modal attention flow, effectively rescuing visually salient anchors. Extensive experiments on Qwen2-VL and Llama-3.2 families demonstrate that V-Skip achieves a $2.9\times$ speedup with negligible accuracy loss. Specifically, it preserves fine-grained visual details, outperforming other baselines over 30\% on the DocVQA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。