arXiv:2503.10496stat.MLcs.AI2025-03被引 1

让神经网络自动跳过无关输入,实现高精度且可解释的压缩模型。

Explainable Bayesian deep learning through input-skip Latent Binary Bayesian Neural Networks

  • 允许输入直接跳过层或被排除,动态构建简洁网络结构。
  • 在MNIST上仅用935个参数达97%准确率,压缩率超99.9%。
  • 无需额外工具即可提供精确的全局与局部解释,适合需要可解释性的场景。

用人工神经网络建模自然现象常能获得高精度预测,但存在过度参数化问题,导致难以解释并引发不确定性。贝叶斯神经网络通过将权重表示为概率分布,可评估预测不确定性。潜在二值贝叶斯神经网络(LBBNN)进一步处理结构不确定性,通过移除冗余权重实现模型稀疏化。本文提出输入跳过型LBBNN(ISLaB),使协变量可跳至任意后续层或被剔除,简化网络并明确输入对预测的影响。该方法可在合适时学习更简单的结构(如线性或仅截距模型)。相比标准LBBNN,ISLaB显著降低网络密度:小网络压缩率超99%,大网络超99.9%,同时保持高预测精度和不确定性量化能力。例如,在MNIST数据集上,仅使用935个权重即达97%准确率,处于当前神经网络压缩的领先水平。此外,该方法能准确识别真实协变量,并适应系统非线性。核心贡献是引入主动路径,直接在LBBNN框架内实现全局与局部解释,其解释具有理论保证且无需事后外部工具。

原文摘要 · Abstract (English)

Modeling natural phenomena with artificial neural networks (ANNs) often provides highly accurate predictions. However, ANNs often suffer from over-parameterization, complicating interpretation and raising uncertainty issues. Bayesian neural networks (BNNs) address the latter by representing weights as probability distributions, allowing for predictive uncertainty evaluation. Latent binary Bayesian neural networks (LBBNNs) further handle structural uncertainty and sparsify models by removing redundant weights. This article advances LBBNNs by enabling covariates to skip to any succeeding layer or be excluded, simplifying networks and clarifying input impacts on predictions. This further allows us to learn simpler structures (e.g., linear or even constant intercept only models) when appropriate. Furthermore, the input-skip LBBNN (ISLaB) approach reduces network density significantly compared to standard LBBNNs, achieving over 99% reduction for small networks and over 99.9% for larger ones, while still maintaining high predictive accuracy and uncertainty quantification. For example, on MNIST, we reached 97% accuracy and great calibration with just 935 weights, reaching state-of-the-art for compression of neural networks. Furthermore, the proposed method accurately identifies the true covariates and adjusts for system non-linearity. The main contribution is the introduction of active paths, enhancing directly designed global and local explanations within the LBBNN framework. The latter are exact with theoretical guarantees and do not require post hoc external tools.

贝叶斯神经网络可解释性模型压缩稀疏化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。