arXiv:2503.18258cs.LGcs.AI2025-03ICLR被引 8

通过剪枝弱相关数据提升模型鲁棒性,无需人工干预

Severing Spurious Correlations with Data Pruning

  • 识别出少数含弱虚假相关性的样本是问题根源
  • 仅剪除少量训练数据即可显著降低虚假依赖
  • 无需领域知识或人工标注,适合实际部署场景

深度神经网络会学习并依赖训练数据中的虚假相关性,导致在真实世界部署时性能崩溃。现有方法多针对虚假信号远强于核心不变信号的场景,易于检测并处理。本文研究虚假信号较弱的新场景,发现其危害源于极少数含虚假特征的样本。为此提出一种无需领域知识、不需样本级虚假信息标注的数据剪枝技术,可自动识别并移除这些关键异常样本。实验表明该方法在已有可识别虚假信号的设置上也达到当前最优性能。

原文摘要 · Abstract (English)

Deep neural networks have been shown to learn and rely on spurious correlations present in the data that they are trained on. Reliance on such correlations can cause these networks to malfunction when deployed in the real world, where these correlations may no longer hold. To overcome the learning of and reliance on such correlations, recent studies propose approaches that yield promising results. These works, however, study settings where the strength of the spurious signal is significantly greater than that of the core, invariant signal, making it easier to detect the presence of spurious features in individual training samples and allow for further processing. In this paper, we identify new settings where the strength of the spurious signal is relatively weaker, making it difficult to detect any spurious information while continuing to have catastrophic consequences. We also discover that spurious correlations are learned primarily due to only a handful of all the samples containing the spurious feature and develop a novel data pruning technique that identifies and prunes small subsets of the training data that contain these samples. Our proposed technique does not require inferred domain knowledge, information regarding the sample-wise presence or nature of spurious information, or human intervention. Finally, we show that such data pruning attains state-of-the-art performance on previously studied settings where spurious information is identifiable.

虚假相关数据剪枝鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。