用瑞尼熵识别关键图像块，高效剪枝同时保持精度
Rényi Attention Entropy for Patch Pruning
- 基于注意力分布的瑞尼熵判断图像块重要性
- 在细粒度图像识别中减少计算量，精度基本不变
- 可灵活调整剪枝策略，适合资源受限场景
Transformer 在视觉与语言任务中表现优异，因其自注意力能捕捉长程依赖。但自注意力计算成本随标记数呈平方增长。通过估计每块重要性并移除冗余块，块剪枝可缓解这一问题。本文提出一种基于注意力分布香农熵的判别准则：低熵块（注意力集中）视为重要保留，高熵块（注意力分散）视为冗余剔除。进一步将该准则扩展至瑞尼熵，强化对尖锐注意力峰值的敏感度，支持根据任务需求和计算限制动态调整剪枝策略。在细粒度图像识别任务中，本方法显著降低计算开销，同时保持模型精度；通过调节瑞尼熵参数，还能进一步优化精度-计算权衡。
原文摘要 · Abstract (English)
Transformers are strong baselines in both vision and language because self-attention captures long-range dependencies across tokens. However, the cost of self-attention grows quadratically with the number of tokens. Patch pruning mitigates this cost by estimating per-patch importance and removing redundant patches. To identify informative patches for pruning, we introduce a criterion based on the Shannon entropy of the attention distribution. Low-entropy patches, which receive selective and concentrated attention, are kept as important, while high-entropy patches with attention spread across many locations are treated as redundant. We also extend the criterion from Shannon to Rényi entropy, which emphasizes sharp attention peaks and supports pruning strategies that adapt to task needs and computational limits. In experiments on fine-grained image recognition, where patch selection is critical, our method reduced computation while preserving accuracy. Moreover, adjusting the pruning policy through the Rényi entropy measure yields further gains and improves the trade-off between accuracy and computation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。