arXiv:2608.17466stat.MLcs.LG2026-08

提出在线稀疏回归新框架,用过参数化提升性能。

Online Generalized Sparse Regression: How Does Overparametrization Help?

论文配图:Online Generalized Sparse Regression: How Does Overparametrization Help?
图 1 · 摘自论文原文
  • 采用约束式稀疏回归,避免动态调参。
  • 算法闭式更新,存储仅需统计量,收敛到最优速率。
  • 适合实时计算场景,对高维数据更有效。

正则化稀疏回归在离线设置下已广泛研究,但在线形式仍相对未被充分探索。这一差距源于四个关键挑战:(i) 在每轮在线更新中动态调整正则化参数不可行;(ii) 存储与内存复杂度难以管理;(iii) 需通过闭式更新实现实时计算,而非每轮求解完整优化问题;(iv) 在合理假设下实现最优统计保证。本文提出一种在线广义稀疏约束回归框架,聚焦于在线基数约束线性回归与低秩矩阵感知。与在线正则化回归不同,我们的约束形式消除了动态参数调优的需求。我们引入一种高效的在线硬阈值算法,实现闭式更新,仅需存储摘要统计量,具备计算、内存与存储效率。尽管该形式存在非凸性与组合性,但在投影集适当过参数化条件下,算法可达到全局收敛且达到最优统计速率。数值实验表明,本方法持续优于现有先进方法。

原文摘要 · Abstract (English)

Regularized sparse regression has been extensively studied in the offline setting, but online formulation remains relatively under-explored. This gap stems from four key challenges: (i) the infeasibility of dynamically updating the regularization parameter in every online round, (ii) managing storage and memory complexity, (iii) enabling real-time computation via closed-form updates rather than solving full optimization problems at each round, and (iv) achieving optimal statistical guarantees under realistic assumptions. In this paper, we propose an online generalized-sparsity-constrained regression framework, focusing on online cardinality-constrained linear regression and low-rank matrix sensing. Unlike online regularized regression, our constrained formulation eliminates the need for dynamic parameter tuning. We introduce an efficient online hard-thresholding algorithm that performs closed-form updates and requires storing only summary statistics, making it computationally, memory, and storage efficient. Despite the inherent nonconvexity and combinatorial nature of the formulation, our algorithm achieves global convergence at the optimal statistical rate under realistic assumptions, provided that the projection set is properly overparameterized. Numerical experiments demonstrate that our method consistently outperforms state-of-the-art alternatives.

稀疏回归在线学习过参数化高效算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。