为最优传输与正则化散度的分布鲁棒优化提供有限样本统计保证。
Statistical Guarantees for Distributionally Robust Optimization with Optimal Transport and OT-Regularized Divergences
- 提出基于最优传输和正则化散度的分布鲁棒优化方法，支持多种代价函数。
- 首次覆盖软约束范数球代价函数，提升对抗训练鲁棒性。
- 适用于对抗样本生成与重加权联合机制，适合关注模型稳健性的研究者。
本文研究了基于最优传输（OT）和OT正则化散度模型邻域的分布鲁棒优化（DRO）在有限样本下的统计性能保证。特别地，我们推导了基于DRO的对抗训练在监督学习中的浓度不等式，该方法常用于增强机器学习模型的对抗鲁棒性。我们的结果适用于广泛的OT代价函数，不仅涵盖以往研究关注的p-Wasserstein情形，更首次包含软约束范数球代价函数——实证表明此类代价能提升对抗训练的鲁棒性；同时，首次将该理论框架扩展至由OT正则化f-散度邻域诱导的对抗样本生成与对抗重加权联合机制，该重加权机制也被证明可进一步提升性能。此外，在p-Wasserstein情形下，我们的边界在分布鲁棒优化邻域大小变化时表现优于先前结果。
原文摘要 · Abstract (English)
We study finite-sample statistical performance guarantees for distributionally robust optimization (DRO) with optimal transport (OT) and OT-regularized divergence model neighborhoods. Specifically, we derive concentration inequalities for supervised learning via DRO-based adversarial training, as commonly employed to enhance the adversarial robustness of machine learning models. Our results apply to a wide range of OT cost functions, beyond the $p$-Wasserstein case studied by previous authors. In particular, our results are the first to: 1) cover soft-constraint norm-ball OT cost functions; soft-constraint costs have been shown empirically to enhance robustness when used in adversarial training, 2) apply to the combination of adversarial sample generation and adversarial reweighting that is induced by using OT-regularized $f$-divergence model neighborhoods; the added reweighting mechanism has also been shown empirically to further improve performance. In addition, even in the $p$-Wasserstein case, our bounds exhibit better behavior as a function of the DRO neighborhood size than previous results when applied to the adversarial setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。