针对稀疏视觉网络修复精度下降问题,提出逐通道自适应修复方法。
Adaptive Signal Resuscitation: Channel-wise Post-Pruning Repair for Sparse Vision Networks

- 按通道粒度动态估算修复系数,避免全局修复过拟合
- 90%稀疏度下在CIFAR-10上恢复55.6%准确率,优于现有方法
- 无需重训练,仅需少量前向计算,适合部署优化
一次性幅度剪枝在高稀疏度下会导致严重精度下降,即使剪枝掩码保留了最大权重。我们认为这源于剪枝后修复的粒度不匹配:同一层中部分通道几乎失效,而另一些仍保持有效激活方差。现有层级激活修复方法对整层应用单一修正,可能过度放大受损通道。我们提出无需训练的逐通道自适应信号复苏(ASR)方法,根据每通道方差匹配估计修正值,并通过数据驱动的收缩规则稳定结果,抑制弱信号通道的不可靠修正,同时保留健康通道的有效修正。ASR在小规模校准集上仅需前向传播,无需重训练。在三个数据集、四种卷积架构及无结构与结构化稀疏设置下,均优于层级修复方法,尤其在高稀疏度下提升显著。在ResNet-50 90%稀疏度下,CIFAR-10上达到55.6% top-1准确率,高于层级修复的41.0%和仅批归一化重校准的28.0%。消融实验表明,朴素的逐通道方差匹配不足,收缩机制对修复稳定性至关重要。
原文摘要 · Abstract (English)
One-shot magnitude pruning can cause severe accuracy collapse in the high-sparsity regime, even when the pruning mask preserves the largest weights. We argue that this failure reflects a granularity mismatch in post-pruning repair. Under global magnitude pruning, nearly collapsed channels can coexist with channels that retain informative activation variance within the same layer. Existing layer-wise activation repair methods apply a single correction to the whole layer, and can therefore over-amplify damaged channels while trying to restore the layer-level signal. We propose Adaptive Signal Resuscitation (ASR), a training-free channel-wise repair method that matches the granularity of repair to the granularity of damage. ASR estimates a variance-matching correction for each output channel and stabilizes it with a data-driven shrinkage rule, suppressing unreliable corrections for channels with weak post-pruning signal while preserving corrections for healthier channels. Applied before BatchNorm recalibration, ASR requires only forward passes on a small calibration set and no retraining. Across three datasets, four convolutional architectures, and both unstructured and structured sparsity settings, ASR generally improves over layer-wise repair, with the clearest gains in high-sparsity regimes. On ResNet-50 at 90% sparsity, ASR recovers 55.6% top-1 accuracy on CIFAR-10, compared with 41.0% for layer-wise repair and 28.0% for BatchNorm-only recalibration. Ablations show that naive channel-wise variance matching is insufficient, and that shrinkage stabilizes post-pruning repair.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。