仅用一次演示+自主交互,让机器人学会新任务
Adaptation of Generalist Robot Policies with Minimal Data

- 先用少量示范锚定预训练模型,再通过在线强化学习优化
- 在1个示范下实现强任务性能,优于现有方法
- 适合需要快速适应新任务的机器人应用
机器人学习的核心目标是从依赖人工数据转向通过自主交互持续改进。然而当前策略仍难以实现完全自主学习:稀疏奖励与弱零样本探索导致机器人很难从零开始发现有效行为。本文研究最小数据适应场景——即预训练机器人策略仅需一个示范和自主在线交互即可学习新任务。该设定是完全自主进化的可行近似,可用于检验少量人类引导是否能启动自主学习及哪些算法成分使其可行。我们提出MiDAS,一种从离线到在线的简单强化学习方案:首先用行为克隆在单次或少数示范上将预训练视觉-语言-动作(VLA)模型锚定到目标任务,随后在残差策略参数化下通过基于价值的在线强化学习进行优化。在LIBERO和RoboCasa数据集上,MiDAS仅凭一个示范即恢复出强任务性能,显著优于基线并实现对演示条件之外场景的泛化。我们在双臂YAM平台上进一步评估,从单次示范获得的低成功率脆弱策略出发,经过约6小时在线交互,MiDAS提升了鲁棒性并学会新成功行为。据我们所知,这是首个可靠实现从单一任务示范中完成机器人策略自适应的案例。
原文摘要 · Abstract (English)
A central goal in robot learning is to move beyond task-specific human data collection toward robots that improve through autonomous interaction. Yet fully autonomous learning remains difficult with current policies: sparse rewards and weak zero-shot exploration make it unlikely that a robot will discover successful behavior from scratch. We study minimal-data adaptation, a regime in which a pre-trained robot policy must learn a new task from as little as one demonstration followed by autonomous online interaction. This setting serves as the closest tractable proxy for fully autonomous improvement, allowing us to study whether minimal human guidance can bootstrap autonomous learning and what algorithmic ingredients make it feasible. We build MiDAS, a simple offline-to-online RL recipe that first anchors a pre-trained VLA to the target task with behavior cloning on single/few demonstrations, then improves it through value-based online RL on a residual policy parameterization. Across LIBERO and RoboCasa, MiDAS recovers strong task performance from as little as one demonstration, substantially outperforming baselines and generalizing beyond demonstrated conditions. We further evaluate MiDAS on a bimanual YAM platform. Starting from a fragile low-success policy obtained from a single demonstration, MiDAS improves its robustness and learns new successful behaviors over ~6 hours of online interaction. To the best of our knowledge, this is the first demonstration of reliable robot policy adaptation from a single task demonstration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。