应对数据分布变化的多目标优化方法，提升模型鲁棒性。
Distributionally Robust Multi-Objective Optimization
- 在最差分布下优化多个目标，增强对数据变化的适应能力。
- 提出双循环和单循环算法，样本复杂度分别达O(ε⁻¹²)和O(ε⁻⁴)。
- 适用于非凸场景，无需有界目标或梯度，适合实际多目标学习任务。
多目标优化（MOO）在需满足多重标准的学习任务中日益重要，但现有方法未显式考虑数据分布变化。本文提出分布鲁棒多目标优化（DR-MOO），在各自最差分布下最小化多个目标。引入帕累托型解概念，并设计具有理论保证的多梯度下降算法（MGDA）。通过拉格朗日对偶重构，先构建双循环MGDA，内层估计对偶变量，达到总样本复杂度O(ε⁻¹²)以达ε-帕累托平稳点。为进一步提升效率，引入梯度裁剪处理广义光滑且有偏梯度估计，消除双采样需求，得到单循环双裁剪MGDA，样本复杂度显著降低至O(ε⁻⁴)。理论适用于非凸情形，无需有界目标或梯度。实验表明，所提方法性能优于现有先进MGDA基线。
原文摘要 · Abstract (English)
Multi-objective optimization (MOO) has received growing attention in applications that require learning under multiple criteria. However, the existing MOO formulations do not explicitly account for distributional shifts in the data. We introduce distributionally robust multi-objective optimization (DR-MOO), which minimizes multiple objectives under their respective worst-case distributions. We propose Pareto-type solution concepts for DR-MOO and develop multi-gradient descent algorithms (MGDA) with provable guarantees. Leveraging a Lagrangian dual reformulation, we first design a double-loop MGDA that uses an inner loop to estimate dual variables and achieves a total sample complexity $\mathcal{O}(ε^{-12})$ for reaching an $ε$-Pareto-stationary point. To further improve efficiency, we incorporate gradient clipping to handle generalized-smooth and biased gradient estimates, removing the need for double sampling. This yields a single-loop double-clip MGDA with substantially improved sample complexity $\mathcal{O}(ε^{-4})$. Our theory applies to the nonconvex setting and does not require bounded objectives or gradients. Experiments demonstrate that our methods are competitive with state-of-the-art MGDA baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。