提出无梯度共识方法求解非凸双层优化问题，理论保证快速收敛。
Convergence of Consensus-Based Particle Methods for Nonconvex Bi-Level Optimization
- 通过平滑分位数选择与吉布斯型近似构建共识点
- 均值场与有限粒子模型均实现指数级收敛至目标解邻域
- 适用于无梯度优化场景，尤其适合复杂神经网络训练
本文研究一种用于非凸双层优化的共识基优化方法，其目标是在低层问题全局最小值集合上最小化高层函数。所提方法无需梯度信息，通过平滑分位数选择结合吉布斯型拉普拉斯近似构造共识点。我们建立了相关均值场动力学及其有限粒子近似的收敛性保证。在平滑分位数定位、误差界和稳定性等合理假设下，证明均值场分布可在任意预设的Wasserstein邻域内以明确指数速率收敛至目标双层解，直至首次命中时间。二维约束问题及神经网络训练的数值实验进一步验证了理论结果。
原文摘要 · Abstract (English)
In this paper, we study a consensus-based optimization method for nonconvex bi-level optimization, where the objective is to minimize an upper-level function over the set of global minimizers of a lower-level problem. The proposed approach is derivative-free, and constructs its consensus point via smooth quantile selection combined with a Gibbs-type Laplace approximation. We establish convergence guarantees for both the associated \textit{mean-field} dynamics and its \textit{finite-particle} approximation. In particular, under suitable assumptions on smooth quantile localization, error bounds, and stability, we show that the mean-field law reaches any arbitrary prescribed Wasserstein neighborhood of the target bi-level solution with an explicit exponential rate up to the hitting time. Numerical experiments on a two-dimensional constrained problem and neural network training further support the theoretical results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。