arXiv:2606.08156cs.CVcs.AI2026-06

根据特征演化层次,动态调整视觉变压器的冗余剪枝与合并策略。

RAPID: Layer-Wise Redundancy-Aware Pruning and Importance-Driven Token Merging for Efficient ViT

论文配图:RAPID: Layer-Wise Redundancy-Aware Pruning and Importance-Driven Token Merging for Efficient ViT
图 1 · 摘自论文原文
  • 分层设计剪枝与合并:浅层用冗余-相似度剪枝局部冗余,深层用重要性-相似度合并保护关键语义
  • 在极端压缩下比ToMe高4.29%准确率,图像分类任务中实现更优压缩-精度权衡
  • 无需训练,可直接部署于ViT和DeiT,适合追求高效推理的模型优化场景

视觉变压器(ViTs)性能强大但因自注意力复杂度为二次方导致计算开销高。虽有剪枝与合并等令牌缩减技术缓解此问题,但通常忽略表征随网络深度演化的特性。本文提出RAPID,一种面向深度的令牌缩减框架,依据各层令牌表征特征自适应调整缩减策略。浅至中层采用冗余-相似度感知剪枝指标,剔除重复的局部模式;深层则转为重要性-相似度感知合并机制,利用分类(CLS)令牌注意力权重保护语义关键令牌,融合不重要且相似的邻居。在ImageNet-1K上对ViT与DeiT架构的实证验证表明,相比即插即用基线如ToMe和ToFu,RAPID在精度-压缩权衡上表现更优,尤其在激进压缩条件下,最高较ToMe提升4.29%准确率。该框架提供了一种无需训练的模型优化模板,使缩减策略与层级特征演化对齐。

原文摘要 · Abstract (English)

Vision Transformers (ViTs) achieve strong performance but suffer from high computational costs due to quadratic self-attention complexity. Although token reduction techniques such as pruning and merging mitigate this, they typically overlook how representations evolve across network depth. We propose RAPID, a depth-aware token reduction framework that adapts reduction strategies to the layer-wise characteristics of token representations. The primary methodological contribution is a bifurcated strategy: in shallow-to-middle layers, RAPID employs a redundancy-similarity aware pruning metric to eliminate over-represented local patterns. As features transition to global semantic concepts in deeper layers, the framework shifts to an importance-similarity aware merging mechanism. This stage leverages classification (CLS) token attention weights to protect semantically critical tokens while fusing less important but similar neighbors. Empirical validation on ImageNet-1K using ViT and DeiT architectures demonstrates that RAPID establishes a superior accuracy-compression Pareto frontier compared to plug-and-play baselines such as ToMe and ToFu. RAPID is particularly robust in aggressive compression regimes, achieving up to 4.29% higher accuracy than ToMe at extreme reduction rates. Our framework provides a training-free template for optimizing vision models by aligning reduction strategies with hierarchical feature evolution.

视觉变压器令牌剪枝模型压缩分层优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。