用函数先验增强贝叶斯神经网络,提升力学代理模型的准确性与不确定性估计。
Empowering Bayesian Neural Networks with Functional Priors through Anchored Ensembling for Mechanics Surrogate Modeling Applications
- 通过锚定集成学习,将低精度模型的知识融入函数空间先验。
- 在多输入多输出材料建模中,外推和分布内数据上均实现高精度与可靠不确定性量化。
- 揭示了权重间相关性对知识迁移的关键作用,改进现有贝叶斯神经网络设计。
近年来,神经网络(NNs)在力学与材料建模的代理模型任务中日益流行。传统神经网络是确定性函数,仅依赖数据学习输入-输出映射;而将其置于贝叶斯框架下可量化不确定性,特别是由训练数据不足引起的认知不确定性,并通过贝叶斯先验整合先验知识。然而,神经网络参数空间维度高且非物理性,参数(权重)与预测输出间关系复杂,导致先验设计与后验推断困难。本文提出一种基于锚定集成的新贝叶斯神经网络(BNN)训练方法,可融合函数空间中的先验信息(如来自低精度模型)。该锚定机制利用预训练中学习到的低秩参数相关性,连接函数空间先验与参数空间。我们还研究了现有BNN中常被忽略的权重相关性,证明其对函数空间与参数空间先验间知识转移至关重要。算法性能首先在1维小规模示例上验证,展示其在插值与外推场景下的行为。随后在多输入多输出材料代理建模任务中进行全面评估,结果表明,无论在分布内还是分布外数据上,本方法在准确性和不确定性估计质量方面均表现优异。
原文摘要 · Abstract (English)
In recent years, neural networks (NNs) have become increasingly popular for surrogate modeling tasks in mechanics and materials modeling applications. While traditional NNs are deterministic functions that rely solely on data to learn the input--output mapping, casting NN training within a Bayesian framework allows to quantify uncertainties, in particular epistemic uncertainties that arise from lack of training data, and to integrate a priori knowledge via the Bayesian prior. However, the high dimensionality and non-physicality of the NN parameter space, and the complex relationship between parameters (NN weights) and predicted outputs, renders both prior design and posterior inference challenging. In this work we present a novel BNN training scheme based on anchored ensembling that can integrate a priori information available in the function space, from e.g. low-fidelity models. The anchoring scheme makes use of low-rank correlations between NN parameters, learnt from pre-training to realizations of the functional prior. We also perform a study to demonstrate how correlations between NN weights, which are often neglected in existing BNN implementations, is critical to appropriately transfer knowledge between the function-space and parameter-space priors. Performance of our novel BNN algorithm is first studied on a small 1D example to illustrate the algorithm's behavior in both interpolation and extrapolation settings. Then, a thorough assessment is performed on a multi--input--output materials surrogate modeling example, where we demonstrate the algorithm's capabilities both in terms of accuracy and quality of the uncertainty estimation, for both in-distribution and out-of-distribution data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。