通过不确定性估计实现高效精准的数据归因，支持大模型和闭源模型。
Daunce: Data Attribution through Uncertainty Estimation
- 基于扰动模型的损失协方差计算数据影响得分，避免依赖梯度。
- 在视觉与大语言模型任务中均优于现有方法，精度更高。
- 首次实现对闭源GPT模型的数据归因，适用于黑盒场景。
训练数据归因（TDA）旨在识别哪些训练样本对特定测试数据的模型预测影响最大。通过量化这些影响，TDA可支持数据调试、清洗与估值等关键应用。基于梯度的TDA方法依赖一阶和二阶信息，难以规模化；虽有随机投影方法提升可扩展性，但常牺牲归因精度。受不确定性与影响函数关联性的启发，我们提出Daunce——一种通过不确定性估计实现数据归因的简单有效方法。该方法通过微调一组扰动模型，并计算各样本损失在模型间的协方差作为归因分数。Daunce可扩展至大语言模型（LLMs），且归因精度高于现有方法。我们在图像任务和LLM微调任务上验证了其有效性，并展示了对黑盒模型的兼容性。应用于OpenAI的GPT模型，本方法首次实现了对专有大模型的数据归因。
原文摘要 · Abstract (English)
Training data attribution (TDA) methods aim to identify which training examples influence a model's predictions on specific test data most. By quantifying these influences, TDA supports critical applications such as data debugging, curation, and valuation. Gradient-based TDA methods rely on gradients and second-order information, limiting their applicability at scale. While recent random projection-based methods improve scalability, they often suffer from degraded attribution accuracy. Motivated by connections between uncertainty and influence functions, we introduce Daunce - a simple yet effective data attribution approach through uncertainty estimation. Our method operates by fine-tuning a collection of perturbed models and computing the covariance of per-example losses across these models as the attribution score. Daunce is scalable to large language models (LLMs) and achieves more accurate attribution compared to existing TDA methods. We validate Daunce on tasks ranging from vision tasks to LLM fine-tuning, and further demonstrate its compatibility with black-box model access. Applied to OpenAI's GPT models, our method achieves, to our knowledge, the first instance of data attribution on proprietary LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。