arXiv:2506.19883cs.LGcs.AI2025-06被引 2

提出新算法STIMULUS,加速多目标优化收敛并减少样本需求。

STIMULUS: Achieving Fast Convergence and Low Sample Complexity in Stochastic Multi-Objective Learning

  • 采用递归框架更新随机梯度估计,提升收敛效率。
  • 非凸场景下样本复杂度达最优 $O(n + \ sqrt{n}ε^{-1})$。
  • 适合追求高效多目标优化的机器学习与工程应用者。

多目标优化(MOO)在机器学习、运筹学和工程领域应用广泛,但现有方法收敛速度慢、样本复杂度高。本文提出一种新算法 STIMULUS(随机路径积分多梯度递归估计器),通过简洁而强大的递归梯度更新机制,在非凸设置下实现 $O(1/T)$ 的收敛率,在强凸设置下达到 $O( ext{exp}(-μT))$。同时,样本复杂度分别达到非凸 $O(n + \sqrt{n}ε^{-1})$ 与强凸 $O(n + \sqrt{n} \ln(μ/ε))$,为当前最优。进一步提出带动量的 STIMULUS-M 及自适应批处理版本 STIMULUS+/STIMULUS-M+,缓解全梯度计算负担,并提供理论分析。

原文摘要 · Abstract (English)

Recently, multi-objective optimization (MOO) has gained attention for its broad applications in ML, operations research, and engineering. However, MOO algorithm design remains in its infancy and many existing MOO methods suffer from unsatisfactory convergence rate and sample complexity performance. To address this challenge, in this paper, we propose an algorithm called STIMULUS( stochastic path-integrated multi-gradient recursive e\ulstimator), a new and robust approach for solving MOO problems. Different from the traditional methods, STIMULUS introduces a simple yet powerful recursive framework for updating stochastic gradient estimates to improve convergence performance with low sample complexity. In addition, we introduce an enhanced version of STIMULUS, termed STIMULUS-M, which incorporates a momentum term to further expedite convergence. We establish $O(1/T)$ convergence rates of the proposed methods for non-convex settings and $O (\exp{-μT})$ for strongly convex settings, where $T$ is the total number of iteration rounds. Additionally, we achieve the state-of-the-art $O \left(n+\sqrt{n}ε^{-1}\right)$ sample complexities for non-convex settings and $O\left(n+ \sqrt{n} \ln ({μ/ε})\right)$ for strongly convex settings, where $ε>0$ is a desired stationarity error. Moreover, to alleviate the periodic full gradient evaluation requirement in STIMULUS and STIMULUS-M, we further propose enhanced versions with adaptive batching called STIMULUS+/ STIMULUS-M+ and provide their theoretical analysis.

多目标优化收敛速率样本复杂度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。