arXiv:2501.08501math.NAcs.LG2025-01被引 4

用新方法让物理信息神经网络更准更快更稳。

Scalable Bayesian Physics-Informed Kolmogorov-Arnold Networks

  • 不用梯度的反演方法,防过拟合且计算快
  • 在大数据和高噪声下仍保持高精度与稳定性
  • 适合需要可靠不确定性估计的大规模科学建模

不确定性量化在科学机器学习中至关重要,尤其当使用代理模型逼近复杂系统时。尽管多层感知机(MLPs)常被用作代理模型,但其参数量大易过拟合。柯尔莫哥洛夫-阿诺德网络(KANs)参数少,是替代方案。然而,基于梯度的推断方法如哈密顿蒙特卡洛(HMC)在处理大规模数据时因反向传播成本高而效率低下。为此,我们提出结合丢弃正则化蒂霍诺夫集合卡尔曼反演(DTEKI)与切比雪夫型KANs的新方法。该无梯度方法有效缓解过拟合并提升数值稳定性。同时引入主动子空间法降低参数空间维度,从而提高预测精度并获得更可靠的不确定性估计。大量实验表明,该方法在多种测试场景中表现优异,包括大数据集和高噪声环境。结果表明,新方法在准确性上可媲美或优于HMC,效率与稳定性显著更高,具备可扩展性;通过利用低维参数子空间,在保持预测精度的同时大幅降低计算成本。

原文摘要 · Abstract (English)

Uncertainty quantification (UQ) plays a pivotal role in scientific machine learning, especially when surrogate models are used to approximate complex systems. Although multilayer perceptions (MLPs) are commonly employed as surrogates, they often suffer from overfitting due to their large number of parameters. Kolmogorov-Arnold networks (KANs) offer an alternative solution with fewer parameters. However, gradient-based inference methods, such as Hamiltonian Monte Carlo (HMC), may result in computational inefficiency when applied to KANs, especially for large-scale datasets, due to the high cost of back-propagation. To address these challenges, we propose a novel approach, combining the dropout Tikhonov ensemble Kalman inversion (DTEKI) with Chebyshev KANs. This gradient-free method effectively mitigates overfitting and enhances numerical stability. Additionally, we incorporate the active subspace method to reduce the parameter-space dimensionality, allowing us to improve the accuracy of predictions and obtain more reliable uncertainty estimates. Extensive experiments demonstrate the efficacy of our approach in various test cases, including scenarios with large datasets and high noise levels. Our results show that the new method achieves comparable or better accuracy, much higher efficiency as well as stability compared to HMC, in addition to scalability. Moreover, by leveraging the low-dimensional parameter subspace, our method preserves prediction accuracy while substantially reducing further the computational cost.

不确定性量化物理信息网络高效推理降维方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。