让模型自动发现并适应不同数据环境,提升泛化能力。
Universal Adaptive Environment Discovery
- 通过学习数据变换分布来动态生成环境,无需预先设定。
- 在多个基准上显著提升最差情况准确率,均值性能不下降。
- 适用于多种鲁棒学习方法,适合追求泛化性能的研究者。
机器学习中的一个开放性问题是如何避免模型利用数据中的虚假相关性;例如,Waterbirds 数据集中的背景-标签捷径。常见方法是在多个环境中训练模型,如随机化背景。但环境选择困难,因通常无法事先获知。我们提出通用自适应环境发现(UAED),一种统一框架,可学习数据变换的分布以生成环境,并优化在该分布上的平均鲁棒目标。UAED 能自动生成 IRM、REx、GroupDRO、CORAL 的自适应版本,无需预定义分组或人工设计环境。理论分析显示其具有 PAC-Bayes 边界,且在标准条件下对测试环境分布具有鲁棒性。实验表明,UAED 发现可解释的环境分布,在标准基准上提升最差情况准确率,同时保持均值性能竞争力。结果表明,使环境自适应是实现分布外泛化的有效路径。
原文摘要 · Abstract (English)
An open problem in Machine Learning is how to avoid models to exploit spurious correlations in the data; a famous example is the background-label shortcut in the Waterbirds dataset. A common remedy is to train a model across multiple environments; in the Waterbirds dataset, this corresponds to training by randomizing the background. However, selecting the right environments is a challenging problem, given that these are rarely known a priori. We propose Universal Adaptive Environment Discovery (UAED), a unified framework that learns a distribution over data transformations that instantiate environments, and optimizes any robust objective averaged over this learned distribution. UAED yields adaptive variants of IRM, REx, GroupDRO, and CORAL without predefined groups or manual environment design. We provide a theoretical analysis by providing PAC-Bayes bounds and by showing robustness to test environment distributions under standard conditions. Empirically, UAED discovers interpretable environment distributions and improves worst-case accuracy on standard benchmarks, while remaining competitive on mean accuracy. Our results indicate that making environments adaptive is a practical route to out-of-distribution generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。