只用最终模型，就能判断数据对模型的影响，方法更统一更可靠。
Final-Model-Only Data Attribution with a Unifying View of Gradient-Based Methods
- 用再训练+平均作为衡量数据影响的黄金标准
- 一阶方法在少量再训练下效果好，但随训练量增加性能下降
- 影响函数方法更稳定但实际精度反而较低
训练数据归因（TDA）旨在通过训练数据理解模型行为。本文关注仅能访问最终训练模型、无法获取训练过程或中间信息的场景。我们将问题重新定义为测量模型对训练样本的敏感性。为此，提出通过适当调整与平均的再训练作为黄金标准方法。进一步地，我们统一了现有基于梯度的TDA方法，证明它们均以不同方式近似该再训练标准。我们在表格、图像和文本数据集及模型上进行了实证研究。结果表明，一阶方法在少量再训练时表现良好，但性能随再训练量增加而下降；而影响函数方法虽更稳定，但近似质量反而更低。
原文摘要 · Abstract (English)
Training data attribution (TDA) is concerned with understanding model behavior in terms of the training data. This paper draws attention to the common setting where one has access only to the final trained model, and not the training algorithm or intermediate information from training. We reframe the problem in this "final-model-only" setting as one of measuring sensitivity of the model to training instances. To operationalize this reframing, we propose further training, with appropriate adjustment and averaging, as a gold standard method to measure sensitivity. We then unify existing gradient-based methods for TDA by showing that they all approximate the further training gold standard in different ways. We investigate empirically the quality of these gradient-based approximations to further training, for tabular, image, and text datasets and models. We find that the approximation quality of first-order methods is sometimes high but decays with the amount of further training. In contrast, the approximations given by influence function methods are more stable but surprisingly lower in quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。