提出超梯度下降法，可保证一维函数全局收敛。
Super Gradient Descent: Global Optimization requires Global Gradient
- 引入全局梯度概念，突破传统算法易陷局部极小的瓶颈。
- 对任意k-Lipschitz函数在闭区间上均能收敛到全局最小值。
- 适合需要精确全局优化的机器学习场景，如线性搜索。
全局最小化是优化中的基础挑战，尤其在机器学习中，直接决定模型性能与收敛性。本文提出一种名为超梯度下降的新优化方法，专用于一维函数，可保证对任意定义在闭区间 [a, b] 上的 k-Lipschitz 函数，均收敛至全局最小值。该方法克服了传统优化算法常陷入局部极小的缺陷，首次引入全局梯度概念，为精确且有指导性的全局优化提供稳健解法。本工作填补了优化理论中的关键空白，在线搜索等具体优化问题中带来新见解与实际进展。
原文摘要 · Abstract (English)
Global minimization is a fundamental challenge in optimization, especially in machine learning, where finding the global minimum of a function directly impacts model performance and convergence. This article introduces a novel optimization method that we called Super Gradient Descent, designed specifically for one-dimensional functions, guaranteeing convergence to the global minimum for any k-Lipschitz function defined on a closed interval [a, b]. Our approach addresses the limitations of traditional optimization algorithms, which often get trapped in local minima. In particular, we introduce the concept of global gradient which offers a robust solution for precise and well-guided global optimization. By focusing on the global minimization problem, this work bridges a critical gap in optimization theory, offering new insights and practical advancements in different optimization problems in particular Machine Learning problems like line search.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。