用随机梯度下降学习单指标模型，能自动适应数据并达到最优学习效率。
Interactive Learning of Single-Index Models via Stochastic Gradient Descent
- 通过自适应学习率，单次梯度下降实现双阶段最优学习。
- 在独立同分布数据下，样本复杂度和累积损失接近理论最优。
- 适合在线学习、自适应数据场景，尤其适用于高维非线性建模。
随机梯度下降（SGD）是高维优化的核心算法，近年来的理论进展揭示了其在高维非线性模型中实现特征学习的能力，尤其在独立同分布数据下的单指标模型中表现突出。本文研究单指标模型的序列学习问题，即广义线性老虎机或岭老虎机问题，其中SGD是一种简单自然的解决方案，但其学习动态尚未被充分理解。我们证明，与最优交互式学习器类似，SGD在此设定下也经历明显的“预热”阶段，随后进入“学习”阶段。此外，采用合适的学习率调度策略，单一SGD过程可同时在两个阶段实现近似最优（或现有最佳）的样本复杂度与累计遗憾保证，适用于广泛类型的连接函数。结果表明，在自适应数据下，SGD对单指标模型的学习依然具有高度竞争力。
原文摘要 · Abstract (English)
Stochastic gradient descent (SGD) is a cornerstone algorithm for high-dimensional optimization, renowned for its empirical successes. Recent theoretical advances have provided a deep understanding of how SGD enables feature learning in high-dimensional nonlinear models, most notably the \textit{single-index model} with i.i.d. data. In this work, we study the sequential learning problem for single-index models, also known as generalized linear bandits or ridge bandits, where SGD is a simple and natural solution, yet its learning dynamics remain largely unexplored. We show that, similar to the optimal interactive learner, SGD undergoes a distinct ``burn-in'' phase before entering the ``learning'' phase in this setting. Moreover, with an appropriately chosen learning rate schedule, a single SGD procedure simultaneously achieves near-optimal (or best-known) sample complexity and regret guarantees across both phases, for a broad class of link functions. Our results demonstrate that SGD remains highly competitive for learning single-index models under adaptive data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。