提出一种仅需单次函数查询的分布式零阶优化方法，适用于不可导场景。
Single Point-Based Distributed Zeroth-Order Optimization with a Non-Convex Stochastic Objective Function
- 基于单点梯度估计实现分布式零阶优化，无需梯度信息。
- 在非凸条件下达到 O(1/∛K) 的收敛速度，优于部分中心化方法。
- 适合无梯度信息、需分布式协作的复杂系统优化任务。
零阶（ZO）优化是应对实际约束的强大工具。梯度追踪（GT）技术被证明是分布式优化中实现共识的有效方法，但它是需要梯度知识的一阶（FO）方法，在实践中并不总可行。本文提出一种基于单点梯度估计的分布式零阶优化方法。我们证明该方法在非凸设置下每次仅需一次噪声函数查询即可收敛。经过 K 次迭代后，收敛速率达 O(1/∛K)，与中心化方法的 O(1/∜K) 相当。最后，数值实验验证了理论结果。
原文摘要 · Abstract (English)
Zero-order (ZO) optimization is a powerful tool for dealing with realistic constraints. On the other hand, the gradient-tracking (GT) technique proved to be an efficient method for distributed optimization aiming to achieve consensus. However, it is a first-order (FO) method that requires knowledge of the gradient, which is not always possible in practice. In this work, we introduce a zero-order distributed optimization method based on a one-point estimate of the gradient tracking technique. We prove that this new technique converges with a single noisy function query at a time in the non-convex setting. We then establish a convergence rate of $O(\frac{1}{\sqrt[3]{K}})$ after a number of iterations K, which competes with that of $O(\frac{1}{\sqrt[4]{K}})$ of its centralized counterparts. Finally, a numerical example validates our theoretical results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。