发现剪枝性能趋同现象，提出按稀疏度匹配特征复杂度的新理论。
Selection Plateau and a Sparsity-Dependent Hierarchy of Pruning Features
- 不同权重评分器在固定稀疏度下精度趋同，源于特征复杂度适配问题。
- 低稀疏度时简单特征即可突破平台，高稀疏度需高阶非单调特征支持。
- 模型设计应根据目标稀疏度动态调整特征复杂度，指导剪枝算法优化。
我们发现单次神经网络剪枝中存在选择平台现象：所有秩单调权重评分器在固定稀疏度下均收敛至相同精度，与函数形式无关。提出稀疏度-信息-复杂度谱（SICS）假说：稀疏度依赖的最小特征复杂度 kappa(S) 决定平台突破，其中 kappa=0 在低稀疏度（S<0.65）已足够，kappa=1 在临界稀疏度（S~0.7）主导，kappa=2 在极端稀疏度（S>0.75）必需。在 ViT-Small/CIFAR-10 上测试九类特征在四个稀疏度下的表现，平滑非单调特征在 S=0.7 时提升 +6.6%，而仅高频波动的原始特征在 S=0.8 时实现 +2.6% 的突破。伪非单调评分器表现低于梯度基线，表明需求为幅度无关的非单调性。手工高斯突起仅带来 +0.006 提升，远低于混沌生成的 +0.046，说明排名对齐必要但不充分。SICS 统一解释了多种剪枝方法的性能聚类现象，并提示未来选择算法应随目标稀疏度自适应特征复杂度。
原文摘要 · Abstract (English)
We identify a Selection Plateau phenomenon in one-shot neural network pruning: all rank-monotone weight scorers converge to identical accuracy at fixed sparsity, independent of functional form. We propose the Sparsity-Information-Complexity Spectrum (SICS) hypothesis: a sparsity-dependent minimum feature complexity kappa(S) governs plateau escape, with kappa=0 sufficient at low sparsity (S<0.65), kappa=1 dominant at critical sparsity (S~0.7), and kappa=2 necessary at extreme sparsity (S>0.75). On ViT-Small/CIFAR-10, testing nine feature classes across four sparsities, smooth non-monotone features provide +6.6% escape at S=0.7, while only raw features with high-frequency wiggle escape at S=0.8 (+2.6%). A fake non-monotone scorer underperforms the gradient baseline, indicating the requirement is magnitude-independent non-monotonicity. A handcrafted Gaussian bump achieves only +0.006 escape vs. chaos-derived +0.046, indicating rank-alignment is necessary but insufficient. SICS provides a unifying explanation for the performance clustering of diverse pruning methods and suggests that future selection algorithms should adapt feature complexity to target sparsity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。