arXiv:2602.19859stat.MLcs.LG2026-02

提出新型贝叶斯神经网络先验,提升模型可解释性与抗攻击能力。

Dirichlet Scale Mixture Priors for Bayesian Neural Networks

  • 引入狄利克雷尺度混合先验,实现结构化稀疏收缩。
  • 在中小规模相关数据上表现更优,参数有效数显著减少。
  • 适合关注模型压缩与鲁棒性的研究者,尤其适用于权重剪枝。

神经网络是现代机器学习的核心,但难以解释、预测过于自信且易受对抗攻击。贝叶斯神经网络(BNNs)虽能缓解部分问题,但先验设定仍具挑战。本文提出一类新型先验——狄利克雷尺度混合(DSM)先验,通过结构化稀疏收缩解决现有局限。理论上推导了其依赖结构与收缩性质,并揭示其在神经网络几何下的表现。实验表明,该先验可促进隐式特征选择,生成稀疏网络,在真实与模拟数据上均对对抗攻击保持鲁棒,且预测性能优异,有效参数显著减少。优势在相关性较强、中等规模数据下尤为明显,更利于权重剪枝。通过重尾收缩机制,还缓解了冷后验效应,为高斯先验提供更合理的替代方案。

原文摘要 · Abstract (English)

Neural networks are the cornerstone of modern machine learning, yet can be difficult to interpret, give overconfident predictions and are vulnerable to adversarial attacks. Bayesian neural networks (BNNs) provide some alleviation of these limitations, but have problems of their own. The key step of specifying prior distributions in BNNs is no trivial task, yet is often skipped out of convenience. In this work, we propose a new class of prior distributions for BNNs, the Dirichlet scale mixture (DSM) prior, that addresses current limitations in Bayesian neural networks through structured, sparsity-inducing shrinkage. Theoretically, we derive general dependence structures and shrinkage results for DSM priors and show how they manifest under the geometry induced by neural networks. In experiments on simulated and real world data we find that the DSM priors encourages sparse networks through implicit feature selection, show robustness under adversarial attacks and deliver competitive predictive performance with substantially fewer effective parameters. In particular, their advantages appear most pronounced in correlated, moderately small data regimes, and are more amenable to weight pruning. Moreover, by adopting heavy-tailed shrinkage mechanisms, our approach aligns with recent findings that such priors can mitigate the cold posterior effect, offering a principled alternative to the commonly used Gaussian priors.

贝叶斯神经网络稀疏性对抗鲁棒性先验设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。