动态捕捉视觉标记重要性变化，避免过早丢弃关键信息
Capturing Token Tendencies for Training-Free Token Pruning in Multimodal Large Language Models
- 通过追踪注意力流动趋势，动态判断标记重要性
- 减少77.8%视觉标记，仅保留约23个最终标记
- 适合追求高效推理且需保留深层语义的多模态模型
尽管视觉标记剪枝对高效多模态大模型至关重要，现有训练无关方法存在根本缺陷：依赖静态、瞬时启发式规则进行不可逆过滤。该方法忽略了多模态大模型的层次结构特性——标记重要性常随层深动态演变而非固定不变。因此，深层推理所需的关键标记常被浅层估计误判并提前丢弃。为此，我们提出趋势感知剪枝（Trend-aware Pruning），将剪枝从局部快照决策提升为时间轨迹建模问题。不依赖孤立得分，而是捕捉注意力流的动量，实现动态修正机制，选择性重激活‘晚发’标记（初始低价值但语义重要性上升）。实验表明，该方法在多种多模态任务中实现更优的效率-性能平衡。显著地，视觉标记减少超过77.8%，最终层仅保留约23个标记，同时保持竞争力，提供一种高效且可逆的多模态推理方案。
原文摘要 · Abstract (English)
While visual token pruning is essential for efficient Multimodal Large Language Models (MLLMs), existing training-free methods suffer from a critical limitation: they rely on static, instantaneous heuristics to perform irreversible filtering. This approach ignores the hierarchical nature of MLLMs, where token importance often evolves dynamically rather than remaining fixed across layers. Consequently, tokens essential for deep-layer reasoning are often prematurely discarded by shallow-layer estimates. To address this, we propose Trend-aware Pruning, a novel framework that elevates pruning from a local snapshot decision to a temporal trajectory modeling problem. Instead of relying on isolated scores, our method captures the momentum of attention flow. This enables a dynamic rectification mechanism that selectively reactivates "late-blooming" tokens, those initially undervalued but exhibiting rising semantic importance, thereby preventing the loss of critical visual cues. Extensive experiments demonstrate that our approach achieves a superior efficiency-performance trade-off across diverse multimodal tasks. Notably, it reduces visual tokens by over 77.8%, retaining only approximately 23 tokens in the final layer while maintaining competitive performance, offering a robust and reversible solution for high-efficiency multimodal inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。