提出稳定在线学习新方法，实现过参数模型在非平稳时间序列上的高效预测。
Adaptive Benign Overfitting (ABO): Overparameterized RLS for Online Learning in Non-stationary Time-series
- 基于QR分解改进递归最小二乘法，防止数值发散。
- 在合成数据上保持残差有界、条件数稳定，复现双下降现象。
- 适合需要快速自适应的金融与能源预测场景。
过参数化模型近期挑战了传统学习理论，展现出超越插值极限的泛化能力，即良性过拟合现象。本文提出自适应良性过拟合（ABO），通过正交三角更新构建数值稳定的递归最小二乘（RLS）框架。引入基于QR分解的指数加权RLS（QR-EWRLS）算法，结合随机傅里叶特征映射与遗忘因子正则化，实现非平稳条件下的在线适应。正交分解避免了协方差形式RLS的数值发散问题，同时保持对演化数据分布的适应性。在非线性合成时间序列上的实验表明，该方法维持有界残差和稳定条件数，并重现过参数模型特有的双下降行为。在外汇与电力需求预测任务中，ABO精度与基线核方法相当，速度提升20%至40%。结果统一了自适应滤波、核近似与良性过拟合，在稳定在线学习框架下建立联系。
原文摘要 · Abstract (English)
Overparameterized models have recently challenged conventional learning theory by exhibiting improved generalization beyond the interpolation limit, a phenomenon known as benign overfitting. This work introduces Adaptive Benign Overfitting (ABO), extending the recursive least-squares (RLS) framework to this regime through a numerically stable formulation based on orthogonal-triangular updates. A QR-based exponentially weighted RLS (QR-EWRLS) algorithm is introduced, combining random Fourier feature mappings with forgetting-factor regularization to enable online adaptation under non-stationary conditions. The orthogonal decomposition prevents the numerical divergence associated with covariance-form RLS while retaining adaptability to evolving data distributions. Experiments on nonlinear synthetic time series confirm that the proposed approach maintains bounded residuals and stable condition numbers while reproducing the double-descent behavior characteristic of overparameterized models. Applications to forecasting foreign exchange and electricity demand show that ABO is highly accurate (comparable to baseline kernel methods) while achieving speed improvements of between 20 and 40 percent. The results provide a unified view linking adaptive filtering, kernel approximation, and benign overfitting within a stable online learning framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。