用自洽数据模型筛选机器人模仿学习数据,提升特定任务表现。
DataMIL: Selecting Data for Robot Imitation Learning with Datamodels
- 基于策略自反馈机制,端到端优化数据选择
- 在60+任务中实现成功率显著提升,最高增益18.7%
- 适用于大规模真实机器人数据集,无需环境试错
近期机器人领域积累了日益庞大且多样的数据集以训练通用策略。然而,这些策略虽在各类任务上平均表现良好,却常在特定任务上表现不佳,需额外调优新任务数据。通过共训练结合任务特定数据与大前序数据集的精心筛选子集,可生成更优专精策略,但盲目选数据反而损害性能。为此,我们提出DataMIL,一种基于datamodels范式的数据选择框架,利用策略自身识别最能提升性能的数据点,实现端到端优化。不同于依赖人工质量判断(如语义或视觉相似性)的标准方法,DataMIL直接针对任务成功优化数据选择,保留有益数据、剔除有害数据。为避免在选择阶段执行昂贵的环境滚动,我们引入任务特定数据的代理损失函数,使DataMIL可在真实世界应用而不降低性能。我们在60多个仿真与真实世界操作任务上验证该方法,尤其在最大开源机器人数据集集合OXE中成功完成数据选择,相比先前工作持续提升成功率。结果强调了端到端、性能感知数据选择对释放大规模先验数据集潜力的重要性。
原文摘要 · Abstract (English)
Recently, the robotics community has amassed ever larger and more diverse datasets to train generalist policies. However, while these policies achieve strong mean performance across a variety of tasks, they often underperform on individual, specialized tasks and require further tuning on newly acquired task-specific data. Combining task-specific data with carefully curated subsets of large prior datasets via co-training can produce better specialized policies, but selecting data naively may actually harm downstream performance. To address this, we introduce DataMIL, a data selection framework built on the datamodels paradigm that reasons about data selection in an end-to-end manner, using the policy itself to identify which data points will most improve performance. Unlike standard practices that filter data using human notions of quality (e.g., based on semantic or visual similarity), DataMIL directly optimizes data selection for task success, allowing us to select data that improves the policy while dropping data that degrade it. To avoid performing expensive rollouts in the environment during selection, we introduce a surrogate loss function on task-specific data, allowing us to use DataMIL in the real world without degrading performance. We validate our approach on 60+ simulation and real-world manipulation tasks, notably showing successful data selection from the largest open collections of robot datasets (OXE); demonstrating consistent gains in success rates over prior works. Our results underscore the importance of end-to-end, performance-aware data selection for unlocking the potential of large prior datasets in robotics. More information at https://robin-lab.cs.utexas.edu/datamodels4imitation/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。