让模型自己选数据,训练快一倍还不丢精度。
Data Agent: Learning to Select Data via End-to-End Dynamic Optimization
- 用强化学习动态决定每轮该用哪些数据
- 在ImageNet-1k上节省超50%训练成本
- 无需调参,能自动适应各种数据场景
动态数据选择旨在通过在线训练中优先选取信息量高的样本以加速训练。然而,现有方法通常依赖于任务特定的手工设计指标或静态/快照式标准来估计样本重要性,限制了其在不同学习范式间的可扩展性,并难以捕捉数据在训练过程中的演化效用。为此,我们提出 Data Agent,一个端到端的动态数据选择框架,将数据选择建模为与训练过程协同演化的序列决策问题。该代理学习一个样本级的选择策略,与模型优化共同进化,由融合损失相关难度与置信度相关不确定性的复合奖励信号驱动。该奖励信号捕捉优化影响与信息增益的互补目标,同时具备免调参的自适应加权机制,可在训练过程中平衡两类信号。在多种数据集和模型架构上的大量实验表明,Data Agent 持续加速训练且保持或提升性能,例如在 ImageNet-1k 和 MMLU 上实现超过50%的成本降低而性能无损。此外,其数据集无关的结构与模块化奖励设计使其可即插即用,适用于各类任务与场景,如对噪声数据的鲁棒性,凸显其在真实场景中的潜力。代码已公开于 https://github.com/Jackbrocp/Data-Agent。
原文摘要 · Abstract (English)
Dynamic Data selection aims to accelerate training by prioritizing informative samples during online training. However, existing methods typically rely on task-specific handcrafted metrics or static/snapshot-based criteria to estimate sample importance, limiting scalability across learning paradigms and making it difficult to capture the evolving utility of data throughout training. To address this challenge, we propose Data Agent, an end-to-end dynamic data selection framework that formulates data selection as a training-aware sequential decision-making problem. The agent learns a sample-wise selection policy that co-evolves with model optimization, guided by a composite reward that integrates loss-based difficulty and confidence-based uncertainty signals. The reward signals capture complementary objectives of optimization impact and information gain, together with a tuning-free adaptive weighting mechanism that balances these signals over training. Extensive experiments across a wide range of datasets and architectures demonstrate that Data Agent consistently accelerates training while preserving or improving performance, e.g., reducing costs by over 50\% on ImageNet-1k and MMLU with lossless performance. Moreover, its dataset-agnostic formulation and modular reward make it plug-and-play across tasks and scenarios, e.g., robustness to noisy datasets, highlighting its potential in real-world scenarios. Code is available at https://github.com/Jackbrocp/Data-Agent.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。