arXiv:2602.00329cs.LGcs.AI2026-02被引 1

为Adam优化器设计了可实时计算的数据贡献度评估方法。

In-Run Data Shapley for Adam Optimizer

  • 基于固定状态假设推导出Adam适配的闭式近似公式
  • 在真实数据上实现接近完美的贡献度精度(相关系数>0.99)
  • 适合需要高效、精准数据溯源的现代深度学习应用

可靠的样本贡献度评估对减少模型偏差和计算浪费至关重要,而Shapley值是理论上的黄金标准。现有‘运行中’方法虽避免了重训练开销,但依赖SGD的线性结构,在Adam等自适应优化器下表现失效。本文证明数据贡献度本质上依赖优化器:基于SGD的代理方法在Adam下与真实贡献差异巨大(皮尔逊相关系数≈0.11)。为此,我们提出面向Adam的运行中数据Shapley方法,通过固定状态假设重构效用函数以恢复可加性,并引入线性化幽灵近似技术,将依赖方差的缩放项线性化,无需显式计算每样本梯度即可高效求解成对梯度点积。大量实验表明,该方法在保持约95%训练吞吐量的同时,实现与真实边际贡献高度一致的精度(相关系数>0.99),并在下游数据溯源任务中显著优于基于SGD的基线。

原文摘要 · Abstract (English)

Reliable data attribution is essential for mitigating bias and reducing computational waste in modern machine learning, with the Shapley value serving as the theoretical gold standard. While recent "In-Run" methods bypass the prohibitive cost of retraining by estimating contributions dynamically, they heavily rely on the linear structure of Stochastic Gradient Descent (SGD) and fail to capture the complex dynamics of adaptive optimizers like Adam. In this work, we demonstrate that data attribution is inherently optimizer-dependent: we show that SGD-based proxies diverge significantly from true contributions under Adam (Pearson $R \approx 0.11$), rendering them ineffective for modern training pipelines. To bridge this gap, we propose Adam-Aware In-Run Data Shapley. We derive a closed-form approximation that restores additivity by redefining utility under a fixed-state assumption and enable scalable computation via a novel Linearized Ghost Approximation. This technique linearizes the variance-dependent scaling term, allowing us to compute pairwise gradient dot-products without materializing per-sample gradients. Extensive experiments show that our method achieves near-perfect fidelity to ground-truth marginal contributions ($R > 0.99$) while retaining $\sim$95\% of standard training throughput. Furthermore, our Adam-aware attribution significantly outperforms SGD-based baselines in data attribution downstream tasks.

数据溯源优化器适配实时评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。