arXiv:2605.21972cs.LG2026-05

不同稀疏分配方式影响模型修复效果,选对方法能显著提升高稀疏度下的恢复精度。

How Sparsity Allocation Shapes Label-Free Post-Pruning Recoverability

  • 比较ERK与LAMP稀疏分配在相同无标签修复流程下的表现
  • 在90%-95.5%稀疏度下,修复准确率差异可达显著水平
  • 适合研究模型压缩与无监督修复的工程师和算法研究员

无结构权重稀疏化在高稀疏度下会使神经网络性能降至接近随机水平,而实际部署中可能无法获得标签用于重训练。无标签后修剪修复方法可部分恢复崩溃的稀疏模型,但其效果取决于上游稀疏分配所留下的结构。本文研究在固定激活统计修复后端条件下,稀疏分配如何影响修复可恢复性。我们在CIFAR-10、CIFAR-100和Imagenette上,使用ResNet-18/34/50,在90%至95.5%稀疏度范围内对比ERK与LAMP分配。结果表明,同一全局稀疏度下,分配方式显著影响修复后准确率,且最优分配随架构、数据集难度和稀疏度变化。我们识别出一个修复敏感过渡区:批归一化重校准开始失效,但激活统计修复仍能恢复非平凡准确率。ImageNet-100与DenseNet-121的额外验证显示,该可恢复区的位置与宽度依赖于数据规模与连接结构。研究提示,稀疏分配与后修剪修复应联合考虑,因分配决定了可用于无标签恢复的激活信号量。

原文摘要 · Abstract (English)

Unstructured magnitude pruning at high sparsity can reduce neural network accuracy to near-random performance, while labeled retraining may be unavailable in practical deployment settings. Label-free post-pruning repair methods can partially recover collapsed sparse models, but their effectiveness depends on the sparse model left by the upstream pruning allocation. This paper studies how sparsity allocation shapes post-repair recoverability under a fixed activation-statistic repair backend. We compare ERK and LAMP allocations under the same label-free repair protocol across CIFAR-10, CIFAR-100, and Imagenette with ResNet-18, ResNet-34, and ResNet-50 at sparsities from 90% to 95.5%. The results show that allocation choice can substantially change post-repair accuracy at the same global sparsity, and that the preferred allocation varies with architecture, dataset difficulty, and sparsity level. We identify a repair-sensitive transition regime in which BatchNorm recalibration begins to fail, while activation-statistic repair still recovers nontrivial accuracy. Additional validation on ImageNet-100 and DenseNet-121 shows that the location and width of this recoverable regime depend on data scale and connectivity structure. These findings suggest that pruning allocation and post-pruning repair should be studied jointly, since the allocation determines how much activation signal remains available for label-free recovery.

模型压缩稀疏化无监督修复神经网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。