提出新方法提升噪声数据下影响函数的可靠性
Towards Robust Influence Functions with Flat Validation Minima
- 基于验证损失平坦性改进影响函数估计
- 在多种任务上显著降低影响估计误差
- 适合关注模型可解释性的研究者使用
影响函数（IF）广泛用于评估单个训练样本对模型预测的影响。然而，现有方法在深度神经网络中，尤其是在噪声训练数据上，往往无法提供可靠的影响力估计。问题根源不在于参数变化估计的不准确（这是以往研究的重点），而在于损失变化估计的缺陷，特别是由于验证风险的尖锐性所致。本文建立了影响估计误差、验证集风险及其尖锐性之间的理论联系，强调平坦验证极小值对准确影响估计的重要性。此外，我们提出了一种专为平坦验证极小值设计的新式影响函数估计方法。跨多种任务的实验结果验证了该方法的优越性。
原文摘要 · Abstract (English)
The Influence Function (IF) is a widely used technique for assessing the impact of individual training samples on model predictions. However, existing IF methods often fail to provide reliable influence estimates in deep neural networks, particularly when applied to noisy training data. This issue does not stem from inaccuracies in parameter change estimation, which has been the primary focus of prior research, but rather from deficiencies in loss change estimation, specifically due to the sharpness of validation risk. In this work, we establish a theoretical connection between influence estimation error, validation set risk, and its sharpness, underscoring the importance of flat validation minima for accurate influence estimation. Furthermore, we introduce a novel estimation form of Influence Function specifically designed for flat validation minima. Experimental results across various tasks validate the superiority of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。