arXiv:2608.26806cs.CV2026-08

不依赖训练的自适应图像令牌剪枝,提升多图处理速度与精度。

Multi-Image Visual Token Pruning in Large Visual Language Models

论文配图:Multi-Image Visual Token Pruning in Large Visual Language Models
图 1 · 摘自论文原文
  • 基于视觉注意力分布动态选择剪枝层,适配不同模型架构。
  • 多图场景下按重要性保留更多令牌,推理速度提升2倍,准确率超94%。
  • 无需训练、兼容高效算子,适合部署在真实多图应用中。

随着真实场景中对多图像序列处理需求的增长,各类视觉令牌剪枝方法应运而生,以缓解大视觉语言模型(LVLMs)面临的计算与上下文长度限制。然而,现有多数剪枝方法依赖静态策略,难以适应不同模型架构和多图像场景,且受限于对注意力计算的依赖,无法兼容FlashAttention等高效技术。为此,我们提出一种无需训练的自适应视觉令牌剪枝(AVTP)框架,适用于多种LVLM架构。通过分析不同LVLM中视觉注意力分布的实证数据,我们战略性地确定剪枝层,并在多图像场景中实现自适应剪枝比例——高重要性图像保留更多令牌。我们在多个LVLM上进行广泛实验,验证了AVTP的有效性与鲁棒性。具体而言,Qwen3VL-8B在多图像基准上实现2倍推理加速,同时保持96.1%原始准确率;InternVL3.5-8B保留94.1%准确率;LLaVA-OV-7B甚至超越原基线性能。代码已开源。

原文摘要 · Abstract (English)

With the growing demand for processing multiple image sequences in real-world applications, various visual token pruning methods have emerged to mitigate the computational and context length constraints faced by Large Vision Language Models (LVLMs). However, most existing pruning approaches rely on static strategies that struggle to adapt across different architectural LVLMs and multi-image scenarios, and are additionally constrained by their dependence on attention computations that are incompatible with efficient techniques like FlashAttention. To address these limitations, we propose a training-free, Adaptive Visual Token Pruning (AVTP) framework, applicable to diverse LVLM architectures. We strategically determine pruning layers based on empirical analysis of visual attention distributions across various LVLMs, and implement adaptive pruning ratios in multi-image contexts where images of higher importance retain proportionally more tokens. We conduct extensive experiments across different LVLMs to demonstrate the effectiveness and robustness of AVTP. Specifically, Qwen3VL-8B achieves 2 times inference speedup while maintaining 96.1\% of its original accuracy on multiple multi-image benchmarks, InternVL3.5-8B retains 94.1\% accuracy, and LLaVA-OV-7B even exceeds its original baseline performance. Our code is available at \href{https://github.com/zry13/AVTP}{this link}.

视觉语言模型令牌剪枝多图处理高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。