给自回归与循环神经网络添加贝叶斯框架，提升不确定性量化能力。
BARNN: A Bayesian Autoregressive and Recurrent Neural Network
- 基于变分丢弃法构建贝叶斯版本模型，兼容大尺寸循环网络。
- 在偏微分方程建模与分子生成任务中精度达或优于现有方法。
- 引入时序变分后验混合先验，实现高效且校准良好的不确定性估计。
自回归与循环神经网络在天气预测、分子生成及大语言模型等领域取得显著进展。然而，这些模型缺乏严谨的不确定性处理框架，而这在偏微分方程求解、分子生成和机器学习力场等科学应用中至关重要。为此，本文提出BARNN：一种基于变分丢弃法的贝叶斯自回归与循环神经网络，可将任意自回归或循环模型转化为其贝叶斯版本。我们还引入时序版“变分后验混合”先验（tVAMP-prior），使贝叶斯推断更高效且校准良好。在偏微分方程建模与分子生成任务上的大量实验表明，BARNN不仅精度达到或超过现有方法，还在不确定性量化与长程依赖建模方面表现优异。
原文摘要 · Abstract (English)
Autoregressive and recurrent networks have achieved remarkable progress across various fields, from weather forecasting to molecular generation and Large Language Models. Despite their strong predictive capabilities, these models lack a rigorous framework for addressing uncertainty, which is key in scientific applications such as PDE solving, molecular generation and Machine Learning Force Fields. To address this shortcoming we present BARNN: a variational Bayesian Autoregressive and Recurrent Neural Network. BARNNs aim to provide a principled way to turn any autoregressive or recurrent model into its Bayesian version. BARNN is based on the variational dropout method, allowing to apply it to large recurrent neural networks as well. We also introduce a temporal version of the "Variational Mixtures of Posteriors" prior (tVAMP-prior) to make Bayesian inference efficient and well-calibrated. Extensive experiments on PDE modelling and molecular generation demonstrate that BARNN not only achieves comparable or superior accuracy compared to existing methods, but also excels in uncertainty quantification and modelling long-range dependencies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。