用方向提示提升零阶优化，自适应地让搜索更准更快。
Adaptively Incorporating Directional Hints into Zeroth-Order Optimization
- 引入控制变量修正零阶梯度估计，动态利用方向提示
- 理论证明收敛速度介于一阶与零阶之间，依赖提示质量
- 无需提前知道提示好坏，实测在复杂场景中持续有效
我们研究在方向提示辅助下的非凸函数零阶优化，方向提示是每轮迭代时由线性子空间给出的廉价但可能不准确的梯度方向近似。为自适应利用这些提示并保持对提示质量的鲁棒性，提出控制变量零阶下降（CV-ZOD）框架，通过基于方向提示设定的控制变量来改进经典零阶梯度估计器。首先证明：最优设置参考向量和步长的预言算法可实现收敛率在 $O(1/T)$（一阶）与 $O(d/T)$（零阶）之间插值，取决于轨迹上提示的质量。随后设计出一种实用变体，无需预先知晓提示质量即可达到该预言保证，仅多出对数因子。在基于模拟的科学优化任务中验证方法，显示在非凸景观下零阶下降更慢，现有引导方法随引导失效而停滞，而本方法仍能持续进步。
原文摘要 · Abstract (English)
We study zeroth-order optimization of non-convex functions with the aid of directional hints, which are cheap but potentially inaccurate approximations of the true gradient direction, given by linear subspaces at each iteration. To leverage these hints adaptively while maintaining robustness to their quality, we introduce Control-Variate Zeroth-Order Descent (CV-ZOD), a new framework that refines the classical zeroth-order gradient estimator with a control variate that can be set based on the directional hints. We first show that the oracle algorithm that optimally sets the reference vector and step size at each iteration achieves a convergence rate that interpolates between the first-order $O(1/T)$ rate and the zeroth-order $O(d/T)$ rate, depending on the quality of the hints along the trajectory. We then develop a practical variant of CV-ZOD that achieves the same oracle guarantee up to logarithmic factors, without any prior knowledge of the hint quality. We validate the method empirically on simulation-based scientific optimization tasks, demonstrating sustained progress on non-convex landscapes where zeroth-order descent is slower and existing guided methods stall as guidance deteriorates.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。