用加速梯度法降低深度贝叶斯网络的计算量，提升训练稳定性。
Improving Infinitely Deep Bayesian Neural Networks with Nesterov's Accelerated Gradient Method
- 引入Nesterov加速梯度优化SDE-BNN，加速收敛。
- 在图像分类和序列建模中减少函数评估次数，提升精度。
- 适合追求高效高精度的深度学习研究者使用。
作为连续深度神经网络的代表性方法，基于随机微分方程（SDE）的贝叶斯神经网络（BNN）因其坚实的理论基础和强大的实际应用潜力而受到广泛关注。然而，其对数值SDE求解器的依赖不可避免地导致大量函数评估（NFE），造成高计算成本并偶尔出现收敛不稳定问题。为解决这些挑战，我们提出一种融合Nesterov加速梯度（NAG）的SDE-BNN模型。通过将NAG融入SDE-BNN框架，并引入与NFE相关的残差跳跃连接，该方法显著加速了收敛过程，大幅减少了训练与测试阶段的NFE。大量实证结果表明，我们的模型在图像分类和序列建模等多种任务中均优于传统SDE-BNN，实现更低的NFE和更高的预测准确率。
原文摘要 · Abstract (English)
As a representative continuous-depth neural network approach, stochastic differential equation (SDE)-based Bayesian neural networks (BNNs) have attracted considerable attention due to their solid theoretical foundations and strong potential for real-world applications. However, their reliance on numerical SDE solvers inevitably incurs a large number of function evaluations (NFEs), resulting in high computational cost and occasional convergence instability. To address these challenges, we propose a Nesterov-accelerated gradient (NAG) enhanced SDE-BNN model. By integrating NAG into the SDE-BNN framework along with an NFE-dependent residual skip connection, our method accelerates convergence and substantially reduces NFEs during both training and testing. Extensive empirical results show that our model consistently outperforms conventional SDE-BNNs across various tasks, including image classification and sequence modeling, achieving lower NFEs and improved predictive accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。