无需重训练即可按需调整预训练ViT大小，实现弹性推理。
Elastic ViTs from Pretrained Models without Retraining
- 通过进化算法近似海森矩阵非对角线结构，结合梯度信息实现高效剪枝。
- 在多种模型上剪枝后性能优于现有方法，5分钟内完成单卡生成。
- 不依赖标签数据，适用于无分类头模型，适合资源受限场景部署。
视觉基础模型表现卓越，但仅支持有限的固定尺寸，导致实际部署中难以适配计算资源限制。本文提出SnapViT：一种无需重训练的单次剪枝方法，可实现视觉变压器在连续计算预算下的弹性推理。该方法通过进化算法近似海森矩阵非对角线结构，融合梯度信息与跨网络结构相关性，无需标签数据，可推广至无分类头模型，且完全免于微调。在DINO、SigLIPv2、DeIT和AugReg等模型上的实验表明，该方法在不同稀疏度下均优于当前最优方案，仅需单张A100 GPU在五分钟内即可生成可任意调节计算量的弹性模型。主要贡献包括针对预训练视觉变压器的高效剪枝策略、一种新型海森矩阵非对角结构的进化近似方法，以及无需标签的自监督重要性评分机制。代码与剪枝模型已公开于：https://elastic.ashita.nl/
原文摘要 · Abstract (English)
Vision foundation models achieve remarkable performance but are only available in a limited set of pre-determined sizes, forcing sub-optimal deployment choices under real-world constraints. We introduce SnapViT: Single-shot network approximation for pruned Vision Transformers, a new post-pretraining structured pruning method that enables elastic inference across a continuum of compute budgets. Our approach efficiently combines gradient information with cross-network structure correlations, approximated via an evolutionary algorithm, does not require labeled data, generalizes to models without a classification head, and is retraining-free. Experiments on DINO, SigLIPv2, DeIT, and AugReg models demonstrate superior performance over state-of-the-art methods across various sparsities, requiring less than five minutes on a single A100 GPU to generate elastic models that can be adjusted to any computational budget. Our key contributions include an efficient pruning strategy for pretrained Vision Transformers, a novel evolutionary approximation of Hessian off-diagonal structures, and a self-supervised importance scoring mechanism that maintains strong performance without requiring retraining or labels. Code and pruned models are available at: https://elastic.ashita.nl/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。