通过自动替换背景提升模型泛化能力,避免依赖错误线索。
Automated Background Swapping for Robustness against Spurious Backgrounds

- 用辅助网络分离前景与背景,再合成新背景进行数据增强。
- 仅需几百张图像标注即可训练,显著提升模型在复杂场景下的表现。
- 适用于无破坏性样本的极端情况,适合追求鲁棒性的研究者。
基于深度神经网络的分类器在跨域任务中表现优异,但若依赖训练数据中的虚假相关性(如背景特征与标签强关联但无因果关系),则会严重失准。在视觉任务中,许多此类虚假相关性源于图像背景,而真正预测类别的是前景。本文提出自动化背景替换(AutoBackSwap)方法,通过二级网络分离前景与背景,利用修补技术生成新背景,并将不同前景与修复背景组合以扩充训练数据。实验表明,仅需对数百张样本进行像素级标注,即可训练该二级网络并自动完成全量数据增强。相较于以往方法,AutoBackSwap在无任何破坏性样本的情况下仍表现优异,在多个存在虚假背景的图像分类任务中持续超越现有技术。
原文摘要 · Abstract (English)
Classifiers based on Deep Neural Networks exhibit strong performance across domains, yet can fail catastrophically if they rely on spurious correlations, i.e., features that are predictive of the target label in the training data but are not causally linked and thus fail to generalize. For the vision domain, many such spurious correlations manifest themselves within the background of the image, where only the foreground is predictive of the class label. In this paper, we introduce Automated Background Swapping (AutoBackSwap) to reduce the reliance of classifiers on such spurious backgrounds. AutoBackSwap uses a secondary network to disentangle the foreground and background, followed by infilling to synthesize complete backgrounds, and finally combines different foregrounds and inpainted backgrounds to augment the training data. We find that patch-wise labeling of just a few hundred samples suffices to train the secondary network and automatically augment the full training dataset on challenging image classification tasks. In contrast to many previous methods, AutoBackSwap proves very effective even if there is not a single sample in the training data breaking the spurious correlation. Across a range of image classification tasks with spurious backgrounds, AutoBackSwap consistently outperforms prior methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。