用强化学习选数据,减少冗余提升训练效率
RL-Selector: Reinforcement Learning-Guided Data Selection via Redundancy Assessment
- 基于样本间关系构建冗余度评估,动态指导数据选择
- 在多个数据集和模型上优于当前最优方法,训练更高效
- 适合大规模数据训练场景,尤其关注算力与性能平衡的团队
现代深度模型依赖大规模数据集,但训练成本高昂。真实数据集中存在大量冗余,亟需更高效的数据训练范式。数据选择可通过识别最具代表性样本降低冗余,从而在不损失性能的前提下减少训练开销。现有方法多依赖静态评分或预训练模型,忽视了所选样本在训练过程中的动态演化。本文提出epsilon-sample cover概念,基于样本间关系量化冗余,捕捉数据集内在结构。在此基础上,将数据选择重构为强化学习(RL)问题,设计RL-Selector框架:一个轻量级RL代理利用演化数据分布生成的epsilon-sample cover作为奖励信号,优化选择策略。在多个基准数据集和多样化架构上的实验表明,该方法持续超越现有最先进基线。使用所选数据集训练的模型展现出更强泛化能力与更高训练效率。
原文摘要 · Abstract (English)
Modern deep architectures often rely on large-scale datasets, but training on these datasets incurs high computational and storage overhead. Real-world datasets often contain substantial redundancies, prompting the need for more data-efficient training paradigms. Data selection has shown promise to mitigate redundancy by identifying the most representative samples, thereby reducing training costs without compromising performance. Existing methods typically rely on static scoring metrics or pretrained models, overlooking the combined effect of selected samples and their evolving dynamics during training. We introduce the concept of epsilon-sample cover, which quantifies sample redundancy based on inter-sample relationships, capturing the intrinsic structure of the dataset. Based on this, we reformulate data selection as a reinforcement learning (RL) process and propose RL-Selector, where a lightweight RL agent optimizes the selection policy by leveraging epsilon-sample cover derived from evolving dataset distribution as a reward signal. Extensive experiments across benchmark datasets and diverse architectures demonstrate that our method consistently outperforms existing state-of-the-art baselines. Models trained with our selected datasets show enhanced generalization performance with improved training efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。