arXiv:2504.00564cs.LG2025-04被引 1

用几何中位数提升数据筛选鲁棒性,对抗噪声数据更有效。

Geometric Median Matching for Robust k-Subset Selection from Noisy Data

  • 基于几何中位数设计迭代选择策略,抗异常值能力强。
  • 在高噪声环境下仍保持稳定性能,收敛速度比随机采样快两倍。
  • 适合大规模数据清洗与高噪声场景下的模型训练优化。

数据剪枝——从大规模数据集中选择小而具代表性的子集——对于降低现代深度学习模型训练的计算成本至关重要。由于大规模数据集普遍存在噪声,开发在数据被污染时仍能保持稳健的数据剪枝方法尤为关键。然而,现有方法依赖经验均值估计,对异常值敏感,在高噪声率下表现不佳。为此,我们提出几何中位数(GM)匹配,一种新型k-子集选择策略,利用几何中位数这一具有最优断裂点1/2的鲁棒估计器,增强对噪声数据的抗干扰能力。该方法通过迭代选择一个k-子集,使其均值逼近潜在噪声数据集的几何中位数,确保在任意污染条件下仍具鲁棒性。理论分析表明,GM匹配在任意污染下仍具备改进的O(1/k)收敛速率,相较随机采样有二次级提升。大量实验在图像分类和图像生成任务中验证了其优越性,尤其在高污染率和高剪枝率下表现突出,成为鲁棒数据剪枝的强大基线。

原文摘要 · Abstract (English)

Data pruning -- the combinatorial task of selecting a small and representative subset from a large dataset, is crucial for mitigating the enormous computational costs associated with training data-hungry modern deep learning models at scale. Since large scale data collections are invariably noisy, developing data pruning strategies that remain robust even in the presence of corruption is critical in practice. However, existing data pruning methods often fail under high corruption rates due to their reliance on empirical mean estimation, which is highly sensitive to outliers. In response, we propose Geometric Median (GM) Matching, a novel k-subset selection strategy that leverages Geometric Median -- a robust estimator with an optimal breakdown point of 1/2; to enhance resilience against noisy data. Our method iteratively selects a k-subset such that the mean of the subset approximates the GM of the (potentially) noisy dataset, ensuring robustness even under arbitrary corruption. We provide theoretical guarantees, showing that GM Matching enjoys an improved O(1/k) convergence rate -- a quadratic improvement over random sampling, even under arbitrary corruption. Extensive experiments across image classification and image generation tasks demonstrate that GM Matching consistently outperforms existing pruning approaches, particularly in high-corruption settings and at high pruning rates; making it a strong baseline for robust data pruning.

数据剪枝几何中位数鲁棒学习噪声过滤

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。