通过熵感知机制，精准去除文本噪声并高效压缩视觉标记。
Combating Textual Noise and Redundancy: Entropy-Aware Dense Visual Token Pruning
- 用统计熵识别并过滤文本噪声，提升指令相关性评分精度
- 将选词转化为带空间先验的子模最大化问题，避免冗余
- 在严格标记预算下仍保持细粒度视觉线索，适合高要求多模态任务
视觉标记剪枝是通过压缩冗余图像块来加速视觉语言模型的关键策略，但现有方法在密集指令和细粒度查询下常无法保留关键视觉线索。本文分析其失效原因，发现两大瓶颈：文本噪声广泛分布导致跨模态评分失真，以及标准选词方式引发特征碎片化。为此，提出熵感知密集剪枝（EADP），将剪枝重构为结构化压缩问题。EADP首先利用统计熵量化并过滤文本噪声，获得鲁棒且细粒度的指令相关性评分；随后不采用简单前K选择，而是将标记选择建模为带有空间先验的子模最大化问题，显式保证整体性与非冗余性。大量实验表明，EADP显著改善了视觉语言模型的准确率-效率权衡，在严格标记预算下仍能稳健保留细粒度视觉线索，并在多个挑战性多模态基准上达到当前最优性能。
原文摘要 · Abstract (English)
Visual token pruning is a crucial strategy for accelerating VLMs by compressing redundant image patches, yet existing methods often fail to preserve critical cues under dense instructions and fine-grained queries. In this paper, we investigate this failure and identify two underlying bottlenecks: the widespread dispersion of textual noise that corrupts dense cross-modal scoring, and the feature fragmentation inherent to standard token selection. To address these issues, we propose Entropy-Aware Dense Pruning (EADP), a framework that reformulates pruning as a structured compression problem. EADP first leverages statistical entropy to quantify and filter out textual noise, yielding a robust, fine-grained instruction relevance score. Subsequently, instead of naive Top-K selection, EADP casts token selection as a submodular maximization problem with a spatial prior, explicitly ensuring a holistic and non-redundant visual representation. Extensive experiments demonstrate that EADP improves the accuracy-efficiency trade-off of VLMs, robustly preserving fine-grained visual cues under strict token budgets while achieving SoTA performance on challenging multimodal benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。