arXiv:2606.28652stat.MLcs.LG2026-06

提出自适应硬阈值法,高效处理高维在线分位数回归。

Adaptive Iterative Hard Thresholding for Online High-dimensional Quantile Regression

论文配图:Adaptive Iterative Hard Thresholding for Online High-dimensional Quantile Regression
图 1 · 摘自论文原文
  • 分两阶段更新:前期延迟阈值以积累弱信号,后期频繁投影稳定结果。
  • 在滑动窗口目标下实现对数级累积误差,优于传统方法。
  • 适合高维、非光滑、重尾噪声等复杂在线学习场景。

在线高维回归需算法能顺序更新并保持结构稀疏性。我们提出自适应迭代硬阈值(AIHT)框架,通过交替随机次梯度更新与自适应调度的硬阈值步骤实现。核心思想是分离支持集发现与局部优化:早期延迟阈值,使弱但有信息的坐标有时间积累信号;后期提高投影频率,稳定稀疏估计并利用局部曲率。建立了高维在线分位数回归的理论,该场景中损失函数非光滑,数据可能存在异质性或重尾噪声。在受限曲率与梯度泄漏条件下,AIHT保持在放大稀疏锥内,呈现双阶段收敛行为,并在滑动窗口目标下达到对数级遗憾。模拟实验及阈值调度消融分析验证了机制有效性,表明其优于标准在线稀疏学习基线。

原文摘要 · Abstract (English)

Online high-dimensional regression requires algorithms that can update sequentially while preserving structural sparsity. We propose \textit{Adaptive Iterative Hard Thresholding (AIHT)}, an online sparse-regression framework that alternates stochastic subgradient updates with adaptively scheduled hard-thresholding steps. The key idea is to separate support discovery from local refinement: early in the learning process, AIHT delays thresholding so that weak but informative coordinates have time to accumulate signal, while later it increases the projection frequency to stabilize the sparse estimator and exploit local curvature. We develop the theory for high-dimensional online quantile regression, a challenging setting in which the loss is nonsmooth and the data may exhibit heterogeneity or heavy-tailed noise. Under restricted curvature and gradient-leakage conditions, AIHT remains in an inflated sparse cone, exhibits a two-phase convergence behavior, and attains logarithmic regret for the sliding-window objective. Simulations for online quantile regression, together with threshold-scheduling ablations, support the proposed mechanism and illustrate its advantage over standard online sparse-learning baselines.

在线学习稀疏回归分位数回归自适应阈值

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。