arXiv:2505.03809cs.LGcs.AI2025-05ICML被引 9

动态选数据+增强融合,训练省一半成本还更稳。

When Dynamic Data Selection Meets Data Augmentation

  • 联合优化选样与增强,按样本质量智能筛选
  • ImageNet-1k上减少50%训练量,性能无损
  • 提升抗噪能力,适合真实场景部署

动态数据选择旨在不损失性能的前提下加速训练。然而,减少训练数据会降低数据多样性,可能影响泛化能力。虽然数据增强被广泛用于提升多样性,但通常未与选择过程协同优化。因此,直接结合两者无法充分发挥其协同效应。为此,我们提出一种全新的在线训练框架,首次将动态数据选择与增强统一起来,实现训练效率与性能的双重提升。该方法估计每个样本的局部密度与多模态语义一致性联合分布,从而有针对性地选取适合增强的样本,同时抑制噪声或模糊样本的引入。这使得在不牺牲模型泛化能力的前提下,显著缩小数据集规模。实验结果表明,我们的方法在多种基准数据集和模型架构上均优于现有最先进方法,例如在ImageNet-1k上减少50%训练成本且性能无损。此外,该方法增强了抗噪能力,提升了模型鲁棒性,强化了其在真实场景中的实用性。

原文摘要 · Abstract (English)

Dynamic data selection aims to accelerate training with lossless performance. However, reducing training data inherently limits data diversity, potentially hindering generalization. While data augmentation is widely used to enhance diversity, it is typically not optimized in conjunction with selection. As a result, directly combining these techniques fails to fully exploit their synergies. To tackle the challenge, we propose a novel online data training framework that, for the first time, unifies dynamic data selection and augmentation, achieving both training efficiency and enhanced performance. Our method estimates each sample's joint distribution of local density and multimodal semantic consistency, allowing for the targeted selection of augmentation-suitable samples while suppressing the inclusion of noisy or ambiguous data. This enables a more significant reduction in dataset size without sacrificing model generalization. Experimental results demonstrate that our method outperforms existing state-of-the-art approaches on various benchmark datasets and architectures, e.g., reducing 50\% training costs on ImageNet-1k with lossless performance. Furthermore, our approach enhances noise resistance and improves model robustness, reinforcing its practical utility in real-world scenarios.

数据选择数据增强训练加速模型鲁棒

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。