通过师生模型对抗学习，让模型自动避开数据中的虚假关联。
UnLearning from Experience to Avoid Spurious Correlations
- 用师生模型并行训练，教师根据学生错误反向修正自身。
- 在多个数据集上提升最差组准确率，最高达44.2%。
- 无需分组标签，适合解决数据偏差问题的研究者。
尽管深度神经网络在诸多任务中达到顶尖性能，但其实际表现远比表面脆弱，容易学习训练数据中的虚假相关性，导致意外失效。本文提出一种新方法——从经验中遗忘（UnLearning from Experience, ULE），通过并行训练的学生与教师模型解决该问题。两者接收相同训练批次，学生模型无约束地学习数据中的虚假相关性；教师模型则在解决同一分类任务的同时，避免学生模型的错误。训练过程中，学生越擅长捕捉虚假关联，教师模型就越趋稳健。教师模型利用学生输出对输入的梯度来消除自身错误。实验表明，该方法在Waterbirds、CelebA、Spawrious和UrbanCars数据集上均有效，相比基线方法，最差组准确率提升最高达29.0%（Waterbirds）、44.2%（CelebA）、29.4%（Spawrious）和43.2%（UrbanCars）。
原文摘要 · Abstract (English)
While deep neural networks can achieve state-of-the-art performance in many tasks, these models are more fragile than they appear. They are prone to learning spurious correlations in their training data, leading to surprising failure cases. In this paper, we propose a new approach that addresses the issue of spurious correlations: UnLearning from Experience (ULE). Our method is based on using two classification models trained in parallel: student and teacher models. Both models receive the same batches of training data. The student model is trained with no constraints and pursues the spurious correlations in the data. The teacher model is trained to solve the same classification problem while avoiding the mistakes of the student model. As training is done in parallel, the better the student model learns the spurious correlations, the more robust the teacher model becomes. The teacher model uses the gradient of the student's output with respect to its input to unlearn mistakes made by the student. We show that our method is effective on the Waterbirds, CelebA, Spawrious and UrbanCars datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。