arXiv:2502.09193cs.LG2025-02KDD

用反事实样本抑制过拟合,提升模型泛化能力。

Countering Overfitting with Counterfactual Examples

  • 通过反事实样本与原样本的间隔约束来正则化训练
  • 在多个数据集上优于传统正则化方法
  • 适合需要提升泛化性的模型训练场景

过拟合是机器学习中的经典问题,表现为模型在新数据上表现不佳。传统缓解方法包括早停、数据增强和正则化。本文发现:模型过拟合程度与其生成有效反事实样本的能力呈正相关——过拟合越严重,越容易为随机输入找到反事实例子。为此,我们提出CF-Reg,在训练损失中引入一种新型正则项,通过确保每个样本与其对应反事实之间存在足够间隔来控制过拟合。在多个数据集和模型上的实验表明,该正则化方法普遍优于现有技术。

原文摘要 · Abstract (English)

Overfitting is a well-known issue in machine learning that occurs when a model struggles to generalize its predictions to new, unseen data beyond the scope of its training set. Traditional techniques to mitigate overfitting include early stopping, data augmentation, and regularization. In this work, we demonstrate that the degree of overfitting of a trained model is correlated with the ability to generate counterfactual examples. The higher the overfitting, the easier it will be to find a valid counterfactual example for a randomly chosen input data point. Therefore, we introduce CF-Reg, a novel regularization term in the training loss that controls overfitting by ensuring enough margin between each instance and its corresponding counterfactual. Experiments conducted across multiple datasets and models show that our counterfactual regularizer generally outperforms existing regularization techniques.

过拟合反事实正则化泛化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。