用生成模型构建鲁棒优化的分布不确定性集，提升模型泛化能力
Distributionally Robust Optimization via Generative Ambiguity Modeling
- 基于生成模型构建兼顾一致性与多样性的分布不确定性集
- 在多个任务中实现优于传统方法的分布外泛化性能
- 适合关注模型鲁棒性与泛化能力的研究者
本文研究分布鲁棒优化（DRO），这是一种提升统计学习与优化鲁棒性和泛化能力的基础框架。有效的不确定性集需在保持与名义分布一致的同时，具备足够多样性以覆盖多种潜在场景，并能导出可计算的DRO解。为此，我们提出基于生成模型的不确定性集，可在名义分布支持域之外捕获多种对抗性分布，同时保持与名义分布的一致性。基于此生成不确定性建模，我们提出生成不确定性集的DRO（GAS-DRO）算法，通过参数化生成模型空间求解内层最大化问题。我们正式建立了GAS-DRO的稳定收敛性能。采用扩散模型实现GAS-DRO，实验证明其在机器学习任务中具有显著优于传统方法的分布外（OOD）泛化表现。
原文摘要 · Abstract (English)
This paper studies Distributionally Robust Optimization (DRO), a fundamental framework for enhancing the robustness and generalization of statistical learning and optimization. An effective ambiguity set for DRO must involve distributions that remain consistent to the nominal distribution while being diverse enough to account for a variety of potential scenarios. Moreover, it should lead to tractable DRO solutions. To this end, we propose generative model-based ambiguity sets that capture various adversarial distributions beyond the nominal support space while maintaining consistency with the nominal distribution. Building on this generative ambiguity modeling, we propose DRO with Generative Ambiguity Set (GAS-DRO), a tractable DRO algorithm that solves the inner maximization over the parameterized generative model space. We formally establish the stationary convergence performance of GAS-DRO. We implement GAS-DRO with a diffusion model and empirically demonstrate its superior Out-of-Distribution (OOD) generalization performance in ML tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。