提出新方法精准评估数据对模型影响，提升噪声样本识别能力
Accumulative SGD Influence Estimation for Data Attribution
- 追踪训练过程中的扰动传播，累积计算每样本影响
- 在多种数据和训练条件下，影响估计更准确，尤其长周期下优势明显
- 适合需要精确数据溯源与清洗的场景，如去噪、数据可信度评估
现代以数据为中心的人工智能需要精确的每个样本影响评估。标准的SGD-IE通过累加每轮的代理值近似剔除一个样本的影响，但忽略了跨轮次的累积效应，导致关键样本排序错误。本文提出ACC-SGD-IE，一种轨迹感知的估计器，能将剔除样本的扰动在整个训练过程中传播，并在每一步更新累积影响状态。在平滑强凸设定下，其误差呈几何级收缩；在平滑非凸情形中，误差界得到收紧，更大的小批量进一步降低常数项。实验表明，在成人数据集、20个新闻组和MNIST上，无论数据干净或带噪声、凸或非凸训练，ACC-SGD-IE均获得更优的影响估计，尤其在长时间训练中表现突出。用于下游数据清洗时，能更可靠地识别噪声样本，使用其清洗后的数据训练出的模型性能优于使用SGD-IE清洗的数据。
原文摘要 · Abstract (English)
Modern data-centric AI needs precise per-sample influence. Standard SGD-IE approximates leave-one-out effects by summing per-epoch surrogates and ignores cross-epoch compounding, which misranks critical examples. We propose ACC-SGD-IE, a trajectory-aware estimator that propagates the leave-one-out perturbation across training and updates an accumulative influence state at each step. In smooth strongly convex settings it achieves geometric error contraction and, in smooth non-convex regimes, it tightens error bounds; larger mini-batches further reduce constants. Empirically, on Adult, 20 Newsgroups, and MNIST under clean and corrupted data and both convex and non-convex training, ACC-SGD-IE yields more accurate influence estimates, especially over long epochs. For downstream data cleansing it more reliably flags noisy samples, producing models trained on ACC-SGD-IE cleaned data that outperform those cleaned with SGD-IE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。