过快衰减学习率会导致稀疏学习陷入结构停滞，即使在低维场景下也难收敛。
Step-Size Decay and Structural Stagnation in Greedy Sparse Learning
- 分析了步长衰减过快时稀疏学习的收敛问题
- 理论证明步长衰减过快会引发残差下界，导致学习停滞
- 适用于研究稀疏学习中步长设计的算法工程师
贪婪算法是稀疏逼近和分阶段学习方法（如匹配追踪、提升法）的核心。已知在一般希尔伯特空间中，步长为 $m^{-α}$ 的幂松弛贪婪算法在 $α>1$ 时可能不收敛。本文从稀疏学习视角重新审视该现象，研究具有可控特征相干性的可实现回归问题，推导出残差范数的显式下界，表明即使在低维稀疏设置下，过快衰减的步长调度也会引发结构性停滞。数值实验验证了理论预测，并揭示了特征相干性的作用。结果为贪婪稀疏学习中的步长设计提供了新见解。
原文摘要 · Abstract (English)
Greedy algorithms are central to sparse approximation and stage-wise learning methods such as matching pursuit and boosting. It is known that the Power-Relaxed Greedy Algorithm with step sizes $m^{-α}$ may fail to converge when $α>1$ in general Hilbert spaces. In this work, we revisit this phenomenon from a sparse learning perspective. We study realizable regression problems with controlled feature coherence and derive explicit lower bounds on the residual norm, showing that over-decaying step-size schedules induce structural stagnation even in low-dimensional sparse settings. Numerical experiments confirm the theoretical predictions and illustrate the role of feature coherence. Our results provide insight into step-size design in greedy sparse learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。