用自加权损失函数高效评估数据重要性，省去重复训练
LossVal: Efficient Data Valuation for Neural Networks
- 在损失函数中嵌入自加权机制，训练时同步计算样本重要性
- 实验显示能有效识别噪声样本，区分有益与有害数据
- 适合大规模数据集，计算成本低，可直接用于实际场景
评估单个训练样本的重要性是机器学习中的关键挑战。传统方法需反复重训模型，移除或保留特定样本，计算开销大且忽略数据点间的依赖关系。我们提出LossVal，一种高效的数据估值方法：通过将自加权机制嵌入交叉熵、均方误差等损失函数，在神经网络训练过程中直接计算样本重要性得分。该方法显著降低计算成本，适用于大规模数据集和实际应用。在多个分类与回归任务上进行的实验表明，LossVal能有效识别噪声样本，并准确区分有益与有害样本。我们分析了LossVal的梯度计算过程，进一步揭示其优势。源代码已公开于 https://github.com/twibiral/LossVal。
原文摘要 · Abstract (English)
Assessing the importance of individual training samples is a key challenge in machine learning. Traditional approaches retrain models with and without specific samples, which is computationally expensive and ignores dependencies between data points. We introduce LossVal, an efficient data valuation method that computes importance scores during neural network training by embedding a self-weighting mechanism into loss functions like cross-entropy and mean squared error. LossVal reduces computational costs, making it suitable for large datasets and practical applications. Experiments on classification and regression tasks across multiple datasets show that LossVal effectively identifies noisy samples and is able to distinguish helpful from harmful samples. We examine the gradient calculation of LossVal to highlight its advantages. The source code is available at: https://github.com/twibiral/LossVal
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。