无光滑性假设下求解随机非凸复合优化问题，提出新驻点定义并实现收敛性证明。
Zeroth-Order Methods for Stochastic Nonconvex Nonsmooth Composite Optimization
- 提出两类无需光滑性的近似驻点概念，适配非光滑机器学习场景。
- 设计两种零阶算法，首次在无光滑假设下获得有限时间收敛结果。
- 适用于正则化ReLU网络等典型非光滑模型，适合优化理论研究者。
本文旨在解决一类随机非凸非光滑复合优化问题。以往的复合优化研究通常要求主要部分满足Lipschitz光滑性或某种弱光滑性条件，这排除了如正则化ReLU网络和稀疏支持矩阵机等机器学习实例。本文聚焦于无任何光滑性假设的随机非凸复合优化问题，提出两类新的近似驻点概念，并分别证明了两种零阶算法在有限时间内收敛至这两类近似驻点。最后通过数值实验验证了算法的有效性。
原文摘要 · Abstract (English)
This work aims to solve a stochastic nonconvex nonsmooth composite optimization problem. Previous works on composite optimization problem requires the major part to satisfy Lipschitz smoothness or some relaxed smoothness conditions, which excludes some machine learning examples such as regularized ReLU network and sparse support matrix machine. In this work, we focus on stochastic nonconvex composite optimization problem without any smoothness assumptions. In particular, we propose two new notions of approximate stationary points for such optimization problem and obtain finite-time convergence results of two zeroth-order algorithms to these two approximate stationary points respectively. Finally, we demonstrate that these algorithms are effective using numerical experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。