Hill-ADAM通过交替优化与反向探索，高效逃离局部极小值。
Explore the Loss space with Hill-ADAM
- 交替执行最小化与最大化，主动探索损失空间
- 在5个损失函数和12个图像调色任务中成功找到全局最优
- 无需随机性，可确定性逃离初始遇到的局部极小
本文提出Hill-ADAM优化器，聚焦于在指定损失景观中逃离局部极小值以逼近全局最小。该方法通过确定性状态空间探索实现，消除了随机算法中梯度更新的不确定性，且极少在首次访问时收敛。论文首先推导出在特定模型状态下ADAM优化器步长的解析近似，进而定义了决定ADAM逃逸能力的关键条件。所提算法通过在误差最小化与最大化之间交替进行，先放大误差以逃离局部极小，再重新最小化，从而实现对损失空间的整体探索，最终推断全局最小值的状态。Hill-ADAM在5个损失函数和12个amber-saturated至 cooler-shade 图像颜色校正实例中进行了测试。
原文摘要 · Abstract (English)
This paper introduces Hill-ADAM. Hill-ADAM is an optimizer with its focus towards escaping local minima in prescribed loss landscapes to find the global minimum. Hill-ADAM escapes minima by deterministically exploring the state space. This eliminates uncertainty from random gradient updates in stochastic algorithms while seldom converging at the first minimum that visits. In the paper we first derive an analytical approximation of the ADAM Optimizer step size at a particular model state. From there define the primary condition determining ADAM limitations in escaping local minima. The proposed optimizer algorithm Hill-ADAM alternates between error minimization and maximization. It maximizes to escape the local minimum and minimizes again afterward. This alternation provides an overall exploration throughout the loss space. This allows the deduction of the global minimum's state. Hill-ADAM was tested with 5 loss functions and 12 amber-saturated to cooler-shade image color correction instances.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。