提出概率框架，让学习型优化算法在非凸非光滑情况下也能高概率收敛到临界点。
A Generalization Result for Convergence in Learning-to-Optimize
- 构建概率化证明框架，将经典优化几何论证移植到学习型优化中。
- 理论证明：在参数化损失类上，学习算法以高概率收敛至临界点。
- 摆脱传统安全防护设计，适用于复杂非光滑非凸优化场景。
学习型优化利用机器学习加速优化算法。尽管实证结果显示其显著优于传统优化方法，但理论保障普遍缺失，导致结果难以可靠保证。尤其在收敛性方面，传统优化的几何论证无法直接用于学习型算法。为此，我们提出一种类似经典优化的概率框架，使几何论证可迁移至学习型优化。基于新证明策略，主定理为参数化类的潜在非光滑、非凸损失函数建立了泛化结果，证明学习优化算法以高概率收敛至临界点。这有效将最坏情况分析推广至概率框架，使学习算法设计无需依赖安全防护机制。
原文摘要 · Abstract (English)
Learning-to-optimize leverages machine learning to accelerate optimization algorithms. While empirical results show tremendous improvements compared to classical optimization algorithms, theoretical guarantees are mostly lacking, such that the outcome cannot be reliably assured. Especially, convergence is hardly studied in learning-to-optimize, because conventional convergence guarantees in optimization are based on geometric arguments, which cannot be applied easily to learned algorithms. Thus, we develop a probabilistic framework that resembles classical optimization and allows for transferring geometric arguments into learning-to-optimize. Based on our new proof-strategy, our main theorem is a generalization result for parametric classes of potentially non-smooth, non-convex loss functions and establishes the convergence of learned optimization algorithms to critical points with high probability. This effectively generalizes the results of a worst-case analysis into a probabilistic framework, and frees the design of the learned algorithm from using safeguards.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。