比较贝叶斯神经网络与混合密度网络,揭示其在不确定性建模中的优劣。
Bayesian Neural Networks vs. Mixture Density Networks: Theoretical and Empirical Insights for Uncertainty-Aware Nonlinear Modeling
- 用统一框架对比两种概率建模方法:基于后验与基于似然的思路。
- MDNs在合成数据和骨龄预测上更擅长捕捉多峰响应和自适应不确定性。
- BNN在数据少时提供更可解释的不确定性,适合小样本场景。
本文研究两种主流的概率神经建模方法:贝叶斯神经网络(BNNs)与混合密度网络(MDNs),用于不确定性感知的非线性回归。BNNs通过在网络参数上施加先验分布来包含认知不确定性,而MDNs直接建模条件输出分布,从而捕捉多模态与异方差的数据生成机制。本文构建了一个统一的理论与实证框架进行比较。理论上,在Hölder光滑条件下,我们推导了收敛速率与误差界,发现由于基于似然的特性,MDNs在KL散度收敛速度上更快;而BNNs因变分推断引入额外近似偏差。实证上,我们在合成非线性数据集和放射学基准数据集(RSNA Pediatric Bone Age Challenge)上评估了两种架构。定量与定性结果表明,MDNs更有效地捕捉多峰响应与自适应不确定性;而BNNs在数据有限时提供更具可解释的认知不确定性。研究澄清了后验驱动与似然驱动概率学习的互补优势,为非线性系统中的不确定性建模提供了指导。
原文摘要 · Abstract (English)
This paper investigates two prominent probabilistic neural modeling paradigms: Bayesian Neural Networks (BNNs) and Mixture Density Networks (MDNs) for uncertainty-aware nonlinear regression. While BNNs incorporate epistemic uncertainty by placing prior distributions over network parameters, MDNs directly model the conditional output distribution, thereby capturing multimodal and heteroscedastic data-generating mechanisms. We present a unified theoretical and empirical framework comparing these approaches. On the theoretical side, we derive convergence rates and error bounds under Hölder smoothness conditions, showing that MDNs achieve faster Kullback-Leibler (KL) divergence convergence due to their likelihood-based nature, whereas BNNs exhibit additional approximation bias induced by variational inference. Empirically, we evaluate both architectures on synthetic nonlinear datasets and a radiographic benchmark (RSNA Pediatric Bone Age Challenge). Quantitative and qualitative results demonstrate that MDNs more effectively capture multimodal responses and adaptive uncertainty, whereas BNNs provide more interpretable epistemic uncertainty under limited data. Our findings clarify the complementary strengths of posterior-based and likelihood-based probabilistic learning, offering guidance for uncertainty-aware modeling in nonlinear systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。