Cut-ViT通过子空间一致性实现高效任务自适应剪枝，提升模型鲁棒性与性能。
Cut-ViT: Task-Specific Model Pruning via Gram Anchoring Subspace Consistency
- 基于空间与语义的格拉姆锚矩阵构建子空间基，实现跨模型对齐
- 在9个数据集上6项任务达当前最优，剪枝耗时仅需20.9%
- 适合需要高效部署且任务定制化的视觉模型剪枝场景
视觉基础模型的剪枝受到广泛关注。然而，现有方法通常依赖单一数据集上的点对点令牌对齐，存在鲁棒性下降和任务特异性不足的问题。为此，我们提出一种任务特定剪枝流程Cut-ViT。首先从空间与语义角度构建格拉姆锚矩阵，并进行子空间分解以提取对应子空间基。采用基无关与残差约束，在空间与通道维度上对齐原生DINOv3与剪枝后模型的格拉姆子空间，使子网络继承原模型的鲁棒特征表示。此外，设计谱熵适配机制，量化特征流形在空间与通道维度上的信息密度，从而将剪枝目标适配至具体下游任务。实验表明，Cut-ViT在单张A100 GPU上仅需约1分钟即可生成不同稀疏度的子网络，相比以往方法节省90%时间（仅20.9%）与54.5%显存（45.5%），并在九个数据集上的六个任务中达到最先进性能。
原文摘要 · Abstract (English)
Pruning visual foundation models has attracted considerable attention. However, existing methods focus on rigid point-to-point token alignment on a single dataset for pruning, suffering from two limitations: i) robustness degradation, and ii) task-specificity deficiency. To address these limitations, we propose a task-specific pruning pipeline, named Cut-ViT. Specifically, we first construct gram anchoring matrices from both spatial and semantic perspectives, and perform the subspace decomposition to extract the corresponding subspace bases. Basis-agnostic and residual constraints are then adopted to align the gram subspaces between the native and pruned DINOv3 models along spatial and channel dimensions, enabling subnetworks to inherit robust feature representations of native DINOv3. Furthermore, we design spectral entropy adaptation, which quantifies the information density of feature manifolds along spatial and channel dimensions, thereby adapting the pruning objective to specific downstream tasks. Experiments show that Cut-ViT requires approximately one minute on a single A100 GPU to obtain subnetworks at various sparsity levels, using only 20.9% of the time and 45.5% of the GPU memory compared with previous methods, while achieving SOTA performance on six tasks across nine datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。