揭示梯度下降优化曲线凸性的临界步长，发现2/L是梯度不增的极限。
Convexity of Optimization Curves: Local Sharp Thresholds, Robustness Impossibility, and New Counterexamples
- 分析常步长迭代下优化曲线凸性，确定关键阈值
- 证明步长≤1.75/L时曲线必凸，且该值紧致
- 揭示离散与连续动态的联系，适合优化理论研究者
我们研究一阶方法的优化曲线——即常步长迭代产生的序列 {f(x_n)}_{n≥0}——何时具有凸性，等价于前向差分 f(x_n)−f(x_{n+1}) 非增。对于凸 L-光滑函数上的梯度下降（GD），当步长 η ≤ 1.75/L 时，优化曲线始终凸，且该阈值紧致；此外，梯度范数在 η ≤ 2/L 时保持非增。在连续时间（梯度流）下，优化曲线恒为凸。这些结果补充并细化了经典光滑凸优化工具箱，连接了离散与连续动力学，以及最坏情况分析。
原文摘要 · Abstract (English)
We study when the \emph{optimization curve} of first-order methods -- the sequence \${f(x\_n)}*{n\ge0}\$ produced by constant-stepsize iterations -- is convex, equivalently when the forward differences \$f(x\_n)-f(x*{n+1})\$ are nonincreasing. For gradient descent (GD) on convex \$L\$-smooth functions, the curve is convex for all stepsizes \$η\le 1.75/L\$, and this threshold is tight. Moreover, gradient norms are nonincreasing for all \$η\le 2/L\$, and in continuous time (gradient flow) the curve is always convex. These results complement and refine the classical smooth convex optimization toolbox, connecting discrete and continuous dynamics as well as worst-case analyses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。