通过混合少数类特征，让模型不再依赖数据捷径，提升小样本泛化能力。
Mitigating Shortcut Learning with InterpoLated Learning
- 用多数样本与少数类样本的表征插值，打破数据捷径依赖
- 在多个自然语言任务中显著提升少数样本表现，且不损失多数样本准确率
- 适用于编码器、编码器-解码器、解码器等主流模型架构
经验风险最小化（ERM）会诱导模型利用捷径，即训练数据中普遍存在但与任务无关的输入属性与标签之间的虚假相关性。这种依赖导致模型在少数类样本上泛化能力差，因为这些样本中此类相关性不成立。现有缓解捷径的方法多为特定模型设计，调参困难、计算开销大，且无法改善学习到的表征。为此，我们提出一种名为插值学习（InterpoLL）的新方法：将多数类样本的表征与同类别少数类样本中具有抗捷径特性的特征进行插值，从而削弱捷径影响，使模型学到对多数和少数样本都有效的预测特征。在多个自然语言理解任务上的实验表明，InterpoLL 在少数类上的泛化性能优于 ERM 及当前最优的捷径缓解方法，且不影响多数类准确率。值得注意的是，该效果在编码器、编码器-解码器和解码器仅架构中均成立，证明了方法的广泛适用性。
原文摘要 · Abstract (English)
Empirical risk minimization (ERM) incentivizes models to exploit shortcuts, i.e., spurious correlations between input attributes and labels that are prevalent in the majority of the training data but unrelated to the task at hand. This reliance hinders generalization on minority examples, where such correlations do not hold. Existing shortcut mitigation approaches are model-specific, difficult to tune, computationally expensive, and fail to improve learned representations. To address these issues, we propose InterpoLated Learning (InterpoLL) which interpolates the representations of majority examples to include features from intra-class minority examples with shortcut-mitigating patterns. This weakens shortcut influence, enabling models to acquire features predictive across both minority and majority examples. Experimental results on multiple natural language understanding tasks demonstrate that InterpoLL improves minority generalization over both ERM and state-of-the-art shortcut mitigation methods, without compromising accuracy on majority examples. Notably, these gains persist across encoder, encoder-decoder, and decoder-only architectures, demonstrating the method's broad applicability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。