验证集微小变化会扭曲数据价值评估，导致样本评分整体趋近零。
Validation-Induced Shapley Shifts: How Validation Structure Distorts Data Valuation
- 通过引入噪声改变验证集，发现训练样本的Shapley值系统性下降。
- 噪声扰动使样本局部邻域排序变化，导致价值分布被压缩平坦化。
- 提出归一化与边界感知策略，提升数据估值在真实场景中的稳定性。
Shapley值被广泛用于基于其对验证集性能的边际贡献来评估训练数据的价值。现有实践常假设一旦训练数据和模型固定，这些值即稳定。本文揭示了一种系统性漏洞：即使对验证集进行微小改动（如引入噪声），也会引发Shapley值分布的方向性偏移。随着噪声增加，训练样本的Shapley值趋向于零。我们追溯其根源为噪声诱导的邻域重排效应：扰动改变了验证样本与训练样本间的局部排序，使估值景观趋于平坦。基于KNN-Shapley框架，在合成与真实数据上均验证了这种偏移的一致性与可复现性。研究挑战了Shapley值稳定性的假设，揭示了数据估值中的新脆弱性。为此提出归一化与边界感知的验证策略，以缓解此类扭曲，提升机器学习市场中数据估值的鲁棒性与可解释性。
原文摘要 · Abstract (English)
Shapley values are widely used to attribute value to training data based on their marginal contribution to performance on a validation set. Existing practice often assumes these values are stable once the training data and model are fixed. In this work, we uncover a systematic vulnerability: even modest changes to the validation set, such as introducing noises, cause directional shifts in Shapley distributions. As noises are added, Shapley values of training samples compress toward zero. We trace this to a noise-induced neighborhood reshuffling effect: perturbations alter the local rank order between validation and training samples, flattening the valuation landscape. Using the KNN-Shapley framework, we show through synthetic and real data that these shifts are consistent and reproducible. Our findings challenge the assumption of Shapley stability and reveal a new axis of fragility in data valuation. We propose normalization and boundary-aware validation strategies to mitigate these distortions and enable more robust, interpretable valuation in machine learning marketplaces.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。