arXiv:2602.05809cs.CV2026-02被引 1

模仿人看图答题,高效压缩视觉模型的冗余信息。

Focus-Scan-Refine: From Human Visual Perception to Efficient Visual Token Pruning

  • 模拟人类聚焦关键证据、扫描全局、精修上下文的思考流程。
  • 在不增加令牌数量前提下,显著提升模型精度与推理速度。
  • 适用于各类视觉语言模型,无需重新训练,即插即用。

视觉语言模型(VLMs)常生成大量视觉令牌,导致推理延迟和内存占用过高;而无训练令牌剪枝虽具实用性,现有方法在激进压缩下仍难以平衡局部证据与全局上下文。本文提出一种受人类视觉认知启发的即插即用剪枝框架 Focus-Scan-Refine(FSR):首先结合视觉重要性与指令相关性聚焦关键证据,避免偏向视觉显著但与问题无关区域;随后基于聚焦结果扫描互补上下文,选择与焦点差异最大的令牌;最后通过相似性分配与得分加权融合,将邻近信息聚合至扫描锚点,不增加令牌预算。在多个VLM骨干网络和视觉语言基准上的实验表明,FSR在准确率-效率权衡上持续优于现有最先进剪枝方法。代码已开源:https://github.com/ILOT-code/FSR。

原文摘要 · Abstract (English)

Vision-language models (VLMs) often generate massive visual tokens that greatly increase inference latency and memory footprint; while training-free token pruning offers a practical remedy, existing methods still struggle to balance local evidence and global context under aggressive compression. We propose Focus-Scan-Refine (FSR), a human-inspired, plug-and-play pruning framework that mimics how humans answer visual questions: focus on key evidence, then scan globally if needed, and refine the scanned context by aggregating relevant details. FSR first focuses on key evidence by combining visual importance with instruction relevance, avoiding the bias toward visually salient but query-irrelevant regions. It then scans for complementary context conditioned on the focused set, selecting tokens that are most different from the focused evidence. Finally, FSR refines the scanned context by aggregating nearby informative tokens into the scan anchors via similarity-based assignment and score-weighted merging, without increasing the token budget. Extensive experiments across multiple VLM backbones and vision-language benchmarks show that FSR consistently improves the accuracy-efficiency trade-off over existing state-of-the-art pruning methods. The source codes can be found at https://github.com/ILOT-code/FSR.

视觉语言模型令牌剪枝高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。