改进数据溯源方法，让高维模型更准确判断训练样本影响。
Rescaled Influence Functions: Accurate Data Attribution in High Dimension
- 用重缩放技术修正影响函数，提升高维场景下的预测精度。
- 在真实数据集上验证，重缩放后对样本移除效果的估计更接近实际。
- 适合关注模型可解释性与数据安全的研究者使用。
训练数据如何影响模型行为？这是数据溯源要回答的问题。当前主流的数据溯源方法基于影响函数（IF），利用一阶泰勒近似在不重新训练模型的情况下高效预测移除部分样本的影响，广泛应用于机器学习领域。然而，在高维情形下（参数量 ≥ Ω(样本量)），现有方法常不准确，甚至严重低估样本移除的影响，即使对逻辑回归等简单模型也是如此。本文提出重缩放影响函数（RIF），一种可直接替换传统影响函数的新工具，计算开销极小但精度显著提升。我们在多个真实数据集上对比了IF与RIF，结果表明RIF在实践中能更精准地预测样本移除效果，并提供了理论分析解释其优势。最后，我们设计了一类可欺骗IF检测的数据投毒攻击，但RIF能有效识别此类攻击。
原文摘要 · Abstract (English)
How does the training data affect a model's behavior? This is the question we seek to answer with data attribution. The leading practical approaches to data attribution are based on influence functions (IF). IFs utilize a first-order Taylor approximation to efficiently predict the effect of removing a set of samples from the training set without retraining the model, and are used in a wide variety of machine learning applications. However, especially in the high-dimensional regime (# params $\geq Ω($# samples$)$), they are often imprecise and tend to underestimate the effect of sample removals, even for simple models such as logistic regression. We present rescaled influence functions (RIF), a new tool for data attribution which can be used as a drop-in replacement for influence functions, with little computational overhead but significant improvement in accuracy. We compare IF and RIF on a range of real-world datasets, showing that RIFs offer significantly better predictions in practice, and present a theoretical analysis explaining this improvement. Finally, we present a simple class of data poisoning attacks that would fool IF-based detections but would be detected by RIF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。