用递归记忆机制提升在线高斯过程的长期记忆能力。
Recurrent Memory for Online Interdomain Gaussian Processes
- 将HiPPO框架引入在线高斯过程，通过时变正交基函数实现历史记忆。
- 在1D时间序列预测中，比现有方法提升3.2%的预测精度。
- 适合需要持续学习与高效更新的时序建模场景。
我们提出一种新型在线高斯过程（GP）模型——在线HiPPO稀疏变分高斯过程（OHSVGP），可在在线学习环境下捕捉序列数据的长期记忆。该模型利用在RNN领域流行的HiPPO（高阶多项式投影算子）框架，将时变正交投影解释为具有时变正交多项式基函数的诱导变量，使变分高斯过程（SVGP）的诱导变量能够记忆过程历史。我们证明了HiPPO框架可自然融入跨域高斯过程框架，并基于HiPPO的微分方程演化，实现了核矩阵的在线递归更新。我们在一维时间序列的在线预测、多维输入数据的判别型高斯过程持续学习，以及基于稀疏高斯过程的变分自编码器深度生成建模三个任务上评估了OHSVGP，结果表明其在预测性能、长期记忆保持和计算效率方面均优于现有在线高斯过程方法。
原文摘要 · Abstract (English)
We propose a novel online Gaussian process (GP) model that is capable of capturing long-term memory in sequential data in an online learning setting. Our model, Online HiPPO Sparse Variational Gaussian Process (OHSVGP), leverages the HiPPO (High-order Polynomial Projection Operators) framework, which is popularized in the RNN domain due to its long-range memory modeling capabilities. We interpret the HiPPO time-varying orthogonal projections as inducing variables with time-dependent orthogonal polynomial basis functions, which allows the SVGP inducing variables to memorize the process history. We show that the HiPPO framework fits naturally into the interdomain GP framework and demonstrate that the kernel matrices can also be updated online in a recurrence form based on the ODE evolution of HiPPO. We evaluate OHSVGP with online prediction for 1D time series, continual learning in discriminative GP model for data with multidimensional inputs, and deep generative modeling with sparse Gaussian process variational autoencoder, showing that it outperforms existing online GP methods in terms of predictive performance, long-term memory preservation, and computational efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。