通过随机化积分时间，实现优化算法的加速收敛。
Hamiltonian Descent Algorithms for Optimization: Accelerated Rates via Randomized Integration Time
- 用随机积分时间替代固定时间，改进哈密顿优化方法
- 连续与离散版本均达到与Nesterov方法相当的加速率
- 适合追求高效优化的科研与工程应用
我们研究了用于优化的哈密顿流（HF-opt），其通过模拟哈密顿动力学进行一段时间积分后将速度重置为0以降低目标函数，是采样中哈密顿蒙特卡洛算法在优化中的类比。对于短积分时间，HF-opt的收敛速率与梯度下降在强凸和弱凸函数上的速率相同。我们证明，通过在HF-opt中随机化积分时间，得到的随机哈密顿流（RHF）在连续时间下可实现加速收敛，其速率类似于加速梯度流。我们研究了RHF的离散实现——随机哈密顿梯度下降（RHGD）算法。证明在光滑强凸和弱凸函数最小化问题上，RHGD能达到与Nesterov加速梯度下降（AGD）相同的加速收敛速率。数值实验表明，RHGD在所有设置下均与经典加速方法如AGD具有竞争力，并在某些场景中表现更优。
原文摘要 · Abstract (English)
We study the Hamiltonian flow for optimization (HF-opt), which simulates the Hamiltonian dynamics for some integration time and resets the velocity to $0$ to decrease the objective function; this is the optimization analogue of the Hamiltonian Monte Carlo algorithm for sampling. For short integration time, HF-opt has the same convergence rates as gradient descent for minimizing strongly and weakly convex functions. We show that by randomizing the integration time in HF-opt, the resulting randomized Hamiltonian flow (RHF) achieves accelerated convergence rates in continuous time, similar to the rates for the accelerated gradient flow. We study a discrete-time implementation of RHF as the randomized Hamiltonian gradient descent (RHGD) algorithm. We prove that RHGD achieves the same accelerated convergence rates as Nesterov's accelerated gradient descent (AGD) for minimizing smooth strongly and weakly convex functions. We provide numerical experiments to demonstrate that RHGD is competitive with classical accelerated methods such as AGD across all settings and outperforms them in certain regimes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。