arXiv:2605.25508cs.LG2026-05

提出修复能力诊断法,指导高稀疏度剪枝中损伤分布的合理分配。

Relative Repairability: A Calibration-Based Diagnostic for High-Sparsity Post-Pruning Allocation

论文配图:Relative Repairability: A Calibration-Based Diagnostic for High-Sparsity Post-Pruning Allocation
图 1 · 摘自论文原文
  • 基于校准数据构建修复能力诊断指标,评估各层剪枝后损伤可修复性。
  • 在高稀疏度下,该方法在修复能力过渡区显著优于传统剪枝策略。
  • 适用于需精细控制剪枝损伤分布的模型压缩场景,如边缘设备部署。

在极高稀疏度下,神经网络剪枝不仅决定保留哪些权重,还影响剪枝造成的损伤在网络中的分布位置,以及是否可通过固定轻量修复过程恢复。本文通过修复条件稀疏度分配视角研究此问题,提出相对修复能力(RR),一种基于校准数据的诊断方法,比较逐层剪枝引起的原始激活失真与通道级方差匹配修复后的残余失真。RR仅使用无标签校准数据估算修复后仍残留的局部损伤比例。在ResNet18、ResNet34和VGG16-BN模型上,于CIFAR10和CIFAR100数据集上实验表明,RR并非普遍最优的分配准则。其最有效区域位于架构依赖的可修复性过渡带——此时传统结构或幅度优先剪枝策略可靠性下降,但修复恢复能力尚未完全崩溃。在CIFAR100 ResNet18上,精细扫描显示,RR在中间过渡带优于ERK,在上部区域超越LAMP。强制投影层截断的消融实验进一步表明,受限的ERK会过度保护投影层,导致常规卷积层承受过量稀疏度,降低修复后性能。结果表明,高稀疏度剪枝应同时考虑保留权重与可修复损伤的分配。

原文摘要 · Abstract (English)

At very high sparsity, neural network pruning does more than decide which weights remain. It also determines where pruning induced damage is placed across the network, and whether that damage can be recovered by a fixed lightweight repair procedure. We study this problem through the lens of repair conditioned sparsity allocation. We introduce Relative Repairability (RR), a calibration based diagnostic that compares the raw activation distortion caused by layerwise pruning with the residual distortion left after channelwise variance matching repair. RR estimates the fraction of local damage that remains after repair, using only unlabeled calibration data. Across ResNet18, ResNet34, and VGG16 BN on CIFAR10 and CIFAR100, we find that RR is not a universally dominant allocation rule. Instead, it is most useful near an architecture dependent recoverability transition, where standard structural or magnitude based allocation priors begin to lose reliability but post repair recovery has not yet fully collapsed. On CIFAR100 ResNet18, a fine grained sweep shows that RR improves over ERK across the central transition band and surpasses LAMP near the upper part of this band. A projection forced ablation further shows that capped ERK can over protect projection layers, shifting excessive sparsity onto regular convolutions and reducing post repair recovery. These results suggest that high sparsity pruning should allocate not only retained weights, but also repairable damage.

模型剪枝稀疏度优化损伤修复自适应分配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。