一种自适应梯度优化算法，无需调参即可适用于多种凸优化问题。
DADA: Dual Averaging with Distance Adaptation
- 基于对偶平均框架，动态调整系数以适应梯度与迭代距离。
- 在多种光滑性条件下实现最优收敛速率，包括非光滑、高阶光滑等情形。
- 适合不熟悉问题细节的研究者，尤其适用于未知迭代次数或精度的场景。
我们提出一种新型通用梯度方法，用于求解凸优化问题。该算法名为对偶平均与距离自适应（DADA），基于经典的对偶平均框架，根据观测到的梯度和迭代点与初始点之间的距离动态调整系数，从而无需问题相关的参数。DADA 是一种通用算法，可在目标函数在极小值点附近局部增长受控的各类问题中同时适用。具体包括：非光滑Lipschitz函数、Lipschitz-光滑函数、Hölder-光滑函数、具有高阶Lipschitz导数的函数、准自协调函数以及 (L₀, L₁)-光滑函数等。关键优势在于，DADA 可用于无约束和有约束问题，即使定义域无界，也无需预先知道迭代次数或期望精度。
原文摘要 · Abstract (English)
We present a novel universal gradient method for solving convex optimization problems. Our algorithm, Dual Averaging with Distance Adaptation (DADA), is based on the classical scheme of dual averaging and dynamically adjusts its coefficients based on observed gradients and the distance between iterates and the starting point, eliminating the need for problem-specific parameters. DADA is a universal algorithm that simultaneously works for a broad spectrum of problem classes, provided the local growth of the objective function around its minimizer can be bounded. Particular examples of such problem classes are nonsmooth Lipschitz functions, Lipschitz-smooth functions, Hölder-smooth functions, functions with high-order Lipschitz derivative, quasi-self-concordant functions, and $(L_0,L_1)$-smooth functions. Crucially, DADA is applicable to both unconstrained and constrained problems, even when the domain is unbounded, without requiring prior knowledge of the number of iterations or desired accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。