提出可扩展的数据质量评估方法,快速识别低质量数据点。
Chunked Data Shapley: A Scalable Dataset Quality Assessment for Machine Learning
- 将数据分块并用优化采样与单次梯度下降估算每块贡献。
- 相比现有方法加速80至2300倍,仍能准确识别低质数据区域。
- 适合大规模表格数据的质量分析,支持分类与回归任务。
随着可用数据集的规模和多样性持续增长,数据质量评估已成为实现可靠高效机器学习分析的关键。一种现代的游戏论方法是数据沙普利(Data Shapley),用于量化数据集中每个数据点的价值。然而,当前最先进的沙普利计算方法在应用于大规模数据集时仍面临严重挑战,限制了其实际应用。本文提出一种新的数据沙普利方法——分块数据沙普利(Chunked Data Shapley, C-DaSh),用于识别高质量数据元组。该方法将数据集划分为可管理的块,并通过优化子集选择和单次迭代随机梯度下降估计每块贡献,显著降低计算时间,同时保持高质量结果。我们在多种真实世界分类与回归任务上进行了实证基准测试,结果显示C-DaSh在计算效率(提速80倍至2300倍)和检测低质数据区域的准确性方面均优于现有沙普利近似方法。该方法使大规模表格数据集的质量测量成为可能,适用于分类与回归流程。
原文摘要 · Abstract (English)
As the volume and diversity of available datasets continue to increase, assessing data quality has become crucial for reliable and efficient Machine Learning analytics. A modern, game-theoretic approach for evaluating data quality is the notion of Data Shapley which quantifies the value of individual data points within a dataset. State-of-the-art methods to scale the NP-hard Shapley computation also face severe challenges when applied to large-scale datasets, limiting their practical use. In this work, we present a Data Shapley approach to identify a dataset's high-quality data tuples, Chunked Data Shapley (C-DaSh). C-DaSh scalably divides the dataset into manageable chunks and estimates the contribution of each chunk using optimized subset selection and single-iteration stochastic gradient descent. This approach drastically reduces computation time while preserving high quality results. We empirically benchmark our method on diverse real-world classification and regression tasks, demonstrating that C-DaSh outperforms existing Shapley approximations in both computational efficiency (achieving speedups between 80x - 2300x) and accuracy in detecting low-quality data regions. Our method enables practical measurement of dataset quality on large tabular datasets, supporting both classification and regression pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。