研究正则化能否真正消除模型捷径，发现有时会误杀关键特征。
Do regularization methods for shortcut mitigation work as intended?
- 分析正则化抑制捷径的理论机制
- 发现正则化可能过度抑制因果特征
- 给出有效避免误杀的条件建议
缓解模型对训练数据中虚假相关性的依赖（即捷径）仍是提升泛化能力的关键挑战。现有正则化方法旨在通过增强模型泛化能力来应对这一问题。然而，我们证明这些方法有时会过度正则化，意外地压制了因果特征。本文分析了正则化抑制捷径的理论机制，并探索其有效性边界。此外，我们识别出在何种条件下正则化能有效消除捷径而不损害因果特征。通过在合成数据和真实数据集上的实验，我们的全面分析揭示了正则化技术在应对捷径问题上的优势与局限，为构建更鲁棒的模型提供了指导。
原文摘要 · Abstract (English)
Mitigating shortcuts, where models exploit spurious correlations in training data, remains a significant challenge for improving generalization. Regularization methods have been proposed to address this issue by enhancing model generalizability. However, we demonstrate that these methods can sometimes overregularize, inadvertently suppressing causal features along with spurious ones. In this work, we analyze the theoretical mechanisms by which regularization mitigates shortcuts and explore the limits of its effectiveness. Additionally, we identify the conditions under which regularization can successfully eliminate shortcuts without compromising causal features. Through experiments on synthetic and real-world datasets, our comprehensive analysis provides valuable insights into the strengths and limitations of regularization techniques for addressing shortcuts, offering guidance for developing more robust models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。