通过累积查询样本优化模型先验,提升对虚假相关性的鲁棒性。
Cumulative Meta-Learning from Active Learning Queries for Robustness to Spurious Correlations

- 用查询样本元学习模型的归纳偏置,而非仅更新数据集
- 在多个基准上提升少数群体准确率,最高增益29.9%
- 适合关注模型公平性与泛化能力的研究者
现实数据中的虚假相关性导致模型依赖无关特征,影响可靠性、泛化性和公平性。主动学习可通过查询能区分核心特征与虚假特征的样本缓解此问题。但传统方法仅将查询样本追加至标注集,仅更新似然项。在深度学习中,这些有效样本的影响可能被更大标注集稀释,或被过参数化模型记忆。本文提出累积主动元学习(CAML),利用查询样本元学习模型的先验(归纳偏置),指导模型适应方式。每轮主动学习被视为元学习任务:当前标注集作为元训练数据用于适应,新查询批次作为元测试数据评估泛化性。与传统元学习假设任务独立不同,CAML利用主动学习轮次间的序列依赖,持续优化累积归纳偏置。理论上,该累积形式引入了早期元学习偏置与后期查询目标的耦合项,捕捉标准元学习缺失的依赖关系。实验表明,CAML在多个虚假相关性基准和采样策略下均提升少数群体准确率,其中在Dominoes上提升27.8%,Waterbirds上提升29.9%,SpuCo上提升14.3%,CivilComments上提升24.0%。
原文摘要 · Abstract (English)
Spurious correlations in real-world datasets cause machine learning models to rely on irrelevant patterns, undermining reliability, generalization, and fairness. Active learning offers a promising way to address this failure mode by querying informative samples that distinguish core features from spurious ones. However, standard active-learning methods simply append queried examples to the labeled set, effectively updating only the likelihood term. In deep learning regimes, the influence of these informative samples can be diluted by the larger labeled set and memorized by overparameterized models. We propose Cumulative Active Meta-Learning (CAML), an active-learning framework that uses queried examples to meta-learn the prior, or inductive bias, governing how the model adapts. CAML casts each active-learning round as a meta-learning task: the current labeled set serves as meta-train data for adaptation, while the newly queried batch serves as meta-test data for evaluating generalization. Unlike conventional meta-learning, which treats tasks as independent and identically distributed, CAML exploits the sequential dependence between active-learning rounds by maintaining a cumulative inductive bias that is progressively refined. Theoretically, we show that this cumulative formulation introduces interaction terms that couple earlier meta-learned inductive biases with later query-induced objectives, capturing dependencies absent from standard meta-learning. Empirically, CAML improves minority-group accuracy across spurious-correlation benchmarks and acquisition strategies, with gains of up to 27.8% on Dominoes, 29.9% on Waterbirds, 14.3% on SpuCo, and 24.0% on CivilComments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。