提出一种高效在线数据估值方法，动态评估训练样本影响力。
Layer-Aware Influence for Online Data Valuation Estimation
- 基于损失到输出的梯度，设计分层感知的在线估值器。
- 在大模型预训练、微调和图像分类中显著提升估值精度。
- 计算开销低，适合大规模动态数据清洗与筛选。
以数据为中心的学习强调通过精选高质量训练样本提升性能，而非设计新架构。核心挑战在于高效估计训练样本的影响。以往研究主要关注收敛模型上的静态影响，忽略了优化过程中样本影响的动态变化，尤其在深度模型中尤为明显。为应对频繁估值带来的计算负担，我们提出一种分层感知的在线估值方法，仅需损失到输出的梯度，避免参数级和全网络梯度计算，同时保持排名保真度。在大模型预训练、微调和图像分类任务上的大量实验表明，该方法在显著降低时间和内存消耗的同时提升了估值准确性，使动态数据筛选在实践中更加高效可扩展。
原文摘要 · Abstract (English)
Data-centric learning emphasizes curating high-quality training samples to boost performance rather than designing new architectures. A central problem is to estimate the influence of training sample efficiently. Prior studies largely focus on static influence measured on a converged model, overlooking how data valuation dynamically changes during optimization. This omission neglects the dynamic nature of sample influence during optimization, especially in deep models. To address the computational burden of frequent influence estimation, we develop a layer-aware online estimator that requires only loss-to-output gradients. This design avoids parameter-level and full-network gradients while preserving ranking fidelity. Extensive experiments across LLM pretraining, fine-tuning, and image classification show our method improves accuracy with substantially lower time and memory cost, making dynamic data curation efficient and scalable in practice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。