arXiv:2603.18056cs.LG2026-03

压缩神经网络会破坏可解释性,即使模型整体性能不变。

Fundamental Limits of Neural Network Sparsification: Evidence from Catastrophic Interpretability Collapse

  • 逐步减少神经元激活数,测试压缩对可解释性的影响
  • 局部特征可解释性严重下降,死神经元率高达90.6%
  • 数据越复杂,可解释性崩溃越严重,与算法无关

极端神经网络稀疏化(激活减少90%)对机制可解释性构成严峻挑战:压缩后可解释特征是否仍能保留?本文在混合变分自编码器-稀疏自编码器(VAE-SAE)架构中,研究了在严苛容量约束下的特征生存问题。提出一种自适应稀疏化调度框架,将活跃神经元从500逐步降至50,持续50个训练周期。在dSprites和Shapes3D两个基准数据集上,对比Top-k与L1稀疏化方法,发现核心悖论:全局表征质量(互信息差)保持稳定,但局部特征可解释性系统性崩溃。使用Top-k稀疏化时,dSprites上死神经元率达34.4±0.9%,Shapes3D达62.7±1.3%;而采用L1正则化的“软约束”方式,结果更差:分别达41.7±4.4%和90.6±0.5%。延长训练100个周期也无法恢复死神经元,且崩溃模式在所有阈值定义下均稳健。关键发现:可解释性崩溃随数据复杂度上升——在Top-k下,Shapes3D比dSprites多1.8倍死神经元;在L1下则多2.2倍。这些结果表明,可解释性衰减是压缩过程的内在属性,而非特定算法、训练时长或阈值选择所致。

原文摘要 · Abstract (English)

Extreme neural network sparsification (90% activation reduction) presents a critical challenge for mechanistic interpretability: understanding whether interpretable features survive aggressive compression. This work investigates feature survival under severe capacity constraints in hybrid Variational Autoencoder--Sparse Autoencoder (VAE-SAE) architectures. We introduce an adaptive sparsity scheduling framework that progressively reduces active neurons from 500 to 50 over 50 training epochs, and provide empirical evidence for fundamental limits of the sparsification-interpretability relationship. Testing across two benchmark datasets -- dSprites and Shapes3D -- with both Top-k and L1 sparsification methods, our key finding reveals a pervasive paradox: while global representation quality (measured by Mutual Information Gap) remains stable, local feature interpretability collapses systematically. Under Top-k sparsification, dead neuron rates reach $34.4\pm0.9\%$ on dSprites and $62.7\pm1.3\%$ on Shapes3D at k=50. L1 regularization -- a fundamentally different "soft constraint" paradigm -- produces equal or worse collapse: $41.7\pm4.4\%$ on dSprites and $90.6\pm0.5\%$ on Shapes3D. Extended training for 100 additional epochs fails to recover dead neurons, and the collapse pattern is robust across all tested threshold definitions. Critically, the collapse scales with dataset complexity: Shapes3D (RGB, 6 factors) shows $1.8\times$ more dead neurons than dSprites (grayscale, 5 factors) under Top-k and $2.2\times$ under L1. These findings establish that interpretability collapse under sparsification is intrinsic to the compression process rather than an artifact of any particular algorithm, training duration, or threshold choice.

神经网络压缩可解释性稀疏化特征存活

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。