通过探索采样增强数据,让机器人学会从执行偏差中恢复
RESample: A Robust Data Augmentation Framework via Exploratory Sampling for Robotic Manipulation
- 用保守覆盖函数识别真实数据中缺失的失败场景
- 通过探索采样生成带恢复动作的失败轨迹,扩充训练数据
- 在真实任务中提升成功率最高达12%,仅增20%样本
视觉-语言-动作(VLA)模型在大规模模仿学习数据上训练后表现出强大的操作能力。然而,这些数据主要包含成功轨迹,缺乏执行偏离标准示范时所需的修正监督。部署时,物理偏差引发分布偏移,导致策略进入失败状态,但缺少失败恢复数据使策略无法纠正偏差。为此,我们提出覆盖引导的数据增强框架RESample,主动补充失败恢复数据。具体而言,RESample训练一个保守覆盖函数,识别存在于实际数据分布中但未出现在标准成功示范中的失败案例。基于评估出的覆盖差异,进行探索采样,主动获取后续带有恢复动作的探索行为,以扩展训练数据的覆盖范围,加入失败恢复轨迹。使用增强后的轨迹微调策略后,在真实环境中出现偏差时也能实现恢复。在LIBERO基准和真实操作任务上的实验表明,RESample持续提升策略成功率,最高实现12%的绝对增长,且额外样本不超过20%。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models have shown strong manipulation capability when trained with large-scale imitation learning datasets. However, these datasets that predominantly consist of successful trajectories rarely provide the corrective supervision required when execution deviates from standard demonstrations. During deployment, these physical deviations lead to the distributional shift that drives policy to failure scenarios, yet missing failure recovery data prevents policy from correcting these execution deviations. To address this failure recovery problem, we propose a coverage-guided data augmentation framework RESample to actively supplement demonstration datasets for failure recovery. Specifically, to guide the augmentation and generate failure modes that possibly appear in the real world, RESample trains a conservative coverage function to identify failure cases that reside within the actual data distribution but are missing in the standard successful demonstrations. Guided by the evaluated coverage discrepancy, we perform exploratory sampling to actively sample exploration behaviors followed by recovery actions, extending the coverage of training data with failure recovery trajectories. With the augmented trajectory, the refined policy, which deviated in real settings, can recover from failure. Experiments on the LIBERO benchmark and real-world manipulation tasks show that RESample consistently improves policy success rates, achieving up to 12% absolute gain with no more than 20% additional samples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。