arXiv:2507.12262cs.LGcs.AI2025-07

用神经网络动态调整高斯过程核参数,提升模型表达能力。

A Framework for Nonstationary Gaussian Processes with Neural Network Parameters

  • 将核函数参数设为神经网络输出,实现空间自适应非平稳建模
  • 在多个数据集上优于静态模型和变分近似层级模型
  • 可灵活适配不同非平稳核,且支持大规模数据近似

高斯过程因其灵活性和不确定性量化能力,已成为非参数回归的常用工具。然而,其常使用平稳核函数,限制了模型表达力,对许多数据集不适用。本文提出一种框架,采用随特征空间变化的非平稳核函数,将核参数建模为以特征为输入的神经网络输出,并通过链式法则联合训练神经网络与高斯过程。该方法能清晰描述非平稳参数的行为,兼容大规模数据的近似方法。模型灵活,可轻松适配不同非平稳核,无需重设计优化流程。我们在GPyTorch中实现该方法,可在多个机器学习数据集上测试非平稳方差与噪声版本,结果表明其准确率和对数评分均优于静态模型及基于变分推断的层次模型。类似效果也见于仅含非平稳方差的模型。此外,该方法成功恢复了空间数据的非平稳参数结构。

原文摘要 · Abstract (English)

Gaussian processes have become a popular tool for nonparametric regression because of their flexibility and uncertainty quantification. However, they often use stationary kernels, which limit the expressiveness of the model and may be unsuitable for many datasets. We propose a framework that uses nonstationary kernels whose parameters vary across the feature space, modeling these parameters as the output of a neural network that takes the features as input. The neural network and Gaussian process are trained jointly using the chain rule to calculate derivatives. Our method clearly describes the behavior of the nonstationary parameters and is compatible with approximation methods for scaling to large datasets. It is flexible and easily adapts to different nonstationary kernels without needing to redesign the optimization procedure. Our methods are implemented with the GPyTorch library and can be readily modified. We test a nonstationary variance and noise variant of our method on several machine learning datasets and find that it achieves better accuracy and log-score than both a stationary model and a hierarchical model approximated with variational inference. Similar results are observed for a model with only nonstationary variance. We also demonstrate our approach's ability to recover the nonstationary parameters of a spatial dataset.

高斯过程非平稳性神经网络不确定性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。