arXiv:2604.25021cs.LG2026-04

在线回归动态误差分析新方法,提升预测稳定性与适应性。

Dynamic Regret for Online Regression in RKHS via Discounted VAW and Subspace Approximation

  • 用子空间逼近将折扣VAW方法推广至核希尔伯特空间
  • 动态误差界依赖于比较序列的路径长度,可实现快慢两种速率
  • 适用于高斯、解析点积及Matérn核等多类核函数

研究在再生核希尔伯特空间(RKHS)中基于平方损失的在线回归问题,以时变比较序列的动态后悔准则为评价标准。该方法通过有限维子空间近似,将Jacobsen & Cutkosky(2024)提出的有限维折扣VAW方法拓展至无限维场景。针对固定子空间,采用几何网格上的折扣VAW预测器集合。额外近似误差由核截断的统一投影误差控制。提出一种通用正交截断方法:从核的特征展开出发,构造使特征函数正交化的内积,以首若干基函数张成的空间作为近似子空间。该方法应用于多种近似方案:显式特征展开对高斯和解析点积核给出快速率边界;Mercer截断提供谱逼近,根据特征值衰减速度实现快/慢速率动态后悔界;最后将核截断构造应用于Matérn核,取得良好性能。

原文摘要 · Abstract (English)

We study online regression with the square loss in a reproducing kernel Hilbert space under a dynamic regret criterion. The learner is compared with a time-varying comparator sequence, and the bounds depend on its path length in the RKHS norm. The proposed method transfers the finite-dimensional discounted Vovk--Azoury--Warmuth approach of Jacobsen \& Cutkosky (2024) to the RKHS setting by means of finite-dimensional subspace approximations. For a fixed subspace, we run a VAW-based ensemble of discounted VAW forecasters over a geometric grid of discount factors. The additional approximation error is controlled by the uniform projection error of kernel sections. We then introduce a general orthogonal truncation method: starting from a feature expansion of the kernel, we construct the associated RKHS by introducing an inner product that makes the feature functions orthonormal, and then use the spans of the first basis functions as finite-dimensional approximation spaces. The resulting subspace reduction is applied to several approximation schemes. Explicit feature expansions yield fast-regime bounds for Gaussian and analytic dot-product kernels. Mercer truncations provide a spectral approximation method and lead to dynamic regret bounds in fast and slow regimes, depending on the eigenvalue decay. Finally, we study subspaces spanned by kernel sections and apply this construction to Matérn kernels.

在线学习核方法动态后悔子空间近似

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。