揭示过参数化与先验如何共同塑造贝叶斯神经网络后验分布
On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriors
- 提出冗余引入的三种后验几何重塑机制
- 验证过参数化使权重后验呈现结构化且符合先验分布
- 适合研究贝叶斯深度学习理论的学者参考
贝叶斯神经网络（BNN）后验常被认为不适用于推断，因对称性导致其碎片化、不可识别性加剧维度膨胀，且权重空间先验被视为无意义。本文研究过参数化与先验共同作用下BNN后验的重构机制，揭示冗余引入三大关键现象：平衡性、等概率流形上的权重重分配以及先验一致性。通过远超早期工作的后验采样预算进行广泛实验，验证了过参数化可诱导出结构化且与先验对齐的权重后验分布，深化了对二者相互作用的理解。
原文摘要 · Abstract (English)
Bayesian neural network (BNN) posteriors are often considered impractical for inference, as symmetries fragment them, non-identifiabilities inflate dimensionality, and weight-space priors are seen as meaningless. In this work, we study how overparametrization and priors together reshape BNN posteriors and derive implications allowing us to better understand their interplay. We show that redundancy introduces three key phenomena that fundamentally reshape the posterior geometry: balancedness, weight reallocation on equal-probability manifolds, and prior conformity. We validate our findings through extensive experiments with posterior sampling budgets that far exceed those of earlier works, and demonstrate how overparametrization induces structured, prior-aligned weight posterior distributions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。