提出单循环算法解决弱凸约束优化问题，收敛更快更稳定。
Single-loop Algorithms for Stochastic Non-convex Optimization with Weakly-Convex Constraints
- 用基于hinge的惩罚函数设计单循环算法，避免双循环复杂结构。
- 在弱凸约束下达到最优复杂度，求解近似KKT点效率更高。
- 适用于公平学习和持续学习场景，实验证明有效且鲁棒。
带有多个函数不等式约束的优化在机器学习中具有重要应用。本文研究其中一类关键问题：目标函数与约束函数均为弱凸的情况。现有方法常存在收敛速度慢或依赖双循环算法设计的问题。为此，我们提出一种新型单循环基于惩罚的随机算法。基于经典精确惩罚法，该方法采用hinge型惩罚项，允许使用固定惩罚参数，从而在寻找近似Karush-Kuhn-Tucker（KKT）解方面达到当前最优复杂度。我们进一步将算法扩展至处理有限和耦合复合目标，这类目标广泛存在于人工智能应用中，并在复杂度上优于现有方法。最后，通过在公平学习中的ROC公平性约束和持续学习中的无遗忘约束实验，验证了所提方法的有效性。
原文摘要 · Abstract (English)
Constrained optimization with multiple functional inequality constraints has significant applications in machine learning. This paper examines a crucial subset of such problems where both the objective and constraint functions are weakly convex. Existing methods often face limitations, including slow convergence rates or reliance on double-loop algorithmic designs. To overcome these challenges, we introduce a novel single-loop penalty-based stochastic algorithm. Following the classical exact penalty method, our approach employs a {\bf hinge-based penalty}, which permits the use of a constant penalty parameter, enabling us to achieve a {\bf state-of-the-art complexity} for finding an approximate Karush-Kuhn-Tucker (KKT) solution. We further extend our algorithm to address finite-sum coupled compositional objectives, which are prevalent in artificial intelligence applications, establishing improved complexity over existing approaches. Finally, we validate our method through experiments on fair learning with receiver operating characteristic (ROC) fairness constraints and continual learning with non-forgetting constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。