arXiv:2409.03149stat.MLcs.LG2024-09被引 1

提出可捕捉动态稀疏相关性的多输出高斯过程模型,提升复杂时序数据建模能力。

Non-stationary and Sparsely-correlated Multi-output Gaussian Process with Spike-and-Slab Prior

  • 用时变核函数卷积构建非平稳协方差结构
  • 通过动态高低帽先验自动筛选有效输出关联
  • 适用于高维时序数据,尤其适合存在负迁移的场景

多输出高斯过程(MGP)常被用于迁移学习,以利用多个输出间的共享信息。其优势在于提供预测不确定性量化,对后续决策任务至关重要。然而,传统MGP难以灵活处理具有动态特征的多变量数据,尤其是复杂的时序相关性。此外,当某些输出间无相关性时,盲目迁移可能导致负迁移。为此,本文提出一种非平稳多输出高斯过程模型,可同时捕捉输出间的动态与稀疏相关性。具体地,通过时变核函数的卷积构造MGP的协方差函数,并在相关参数上施加动态高低帽先验,使模型在训练中自动判断哪些源输出对目标输出有信息贡献。采用期望-最大化(EM)算法实现高效拟合。数值实验与真实案例均验证了该模型在捕捉动态稀疏相关结构及缓解负迁移方面的有效性。最后,山车强化学习案例展示了其在非平稳环境下复杂多任务决策中的应用潜力。

原文摘要 · Abstract (English)

Multi-output Gaussian process (MGP) is commonly used as a transfer learning method to leverage information among multiple outputs. A key advantage of MGP is providing uncertainty quantification for prediction, which is highly important for subsequent decision-making tasks. However, traditional MGP may not be sufficiently flexible to handle multivariate data with dynamic characteristics, particularly when dealing with complex temporal correlations. Additionally, since some outputs may lack correlation, transferring information among them may lead to negative transfer. To address these issues, this study proposes a non-stationary MGP model that can capture both the dynamic and sparse correlation among outputs. Specifically, the covariance functions of MGP are constructed using convolutions of time-varying kernel functions. Then a dynamic spike-and-slab prior is placed on correlation parameters to automatically decide which sources are informative to the target output in the training process. An expectation-maximization (EM) algorithm is proposed for efficient model fitting. Both numerical studies and a real case demonstrate its efficacy in capturing dynamic and sparse correlation structure and mitigating negative transfer for high-dimensional time-series data. Finally, a mountain-car reinforcement learning case highlights its potential application in decision making problems.

高斯过程多输出时序建模不确定性量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。