提出新方法缓解子群体偏移中的模型偏差，提升泛化能力。
Curriculum-enhanced GroupDRO: Challenging the Norm of Avoiding Curriculum Learning in Subpopulation Shift Setups
- 先学难样本和易反例，打破早期学习偏差
- 在Waterbirds数据集上提升6.2%准确率
- 适合解决数据分布不均的公平性问题
在子群体偏移场景中，传统课程学习（CL）会过早固化模型对容易识别的伪相关性的依赖。据我们所知，当前最先进的子群体偏移方法均未采用任何课程学习机制。为此，我们设计了一种新的课程学习策略，旨在初始化模型权重于假设空间中更无偏的位置，从而在后续使用全部数据进行优化时避免快速收敛到有偏的假设。本文提出一种增强型组分布鲁棒优化方法（CeGDRO），优先处理最难确认偏见的样本和最易反驳偏见的样本，利用GroupDRO平衡初始难度差异。我们在多个主流子群体偏移数据集上进行了基准测试，结果表明该方法在所有场景下均优于现有最优方法，尤其在Waterbirds数据集上最高提升6.2%。
原文摘要 · Abstract (English)
In subpopulation shift scenarios, a Curriculum Learning (CL) approach would only serve to imprint the model weights, early on, with the easily learnable spurious correlations featured. To the best of our knowledge, none of the current state-of-the-art subpopulation shift approaches employ any kind of curriculum. To overcome this, we design a CL approach aimed at initializing the model weights in an unbiased vantage point in the hypothesis space which sabotages easy convergence towards biased hypotheses during the final optimization based on the entirety of the available data. We hereby propose a Curriculum-enhanced Group Distributionally Robust Optimization (CeGDRO) approach, which prioritizes the hardest bias-confirming samples and the easiest bias-conflicting samples, leveraging GroupDRO to balance the initial discrepancy in terms of difficulty. We benchmark our proposed method against the most popular subpopulation shift datasets, showing an increase over the state-of-the-art results across all scenarios, up to 6.2% on Waterbirds.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。