让机器人像人一样先整理环境再执行任务，提升复杂场景下的操作鲁棒性。
Prepare Before You Act: Learning From Humans to Rearrange Initial States
- 通过分析人类视频与远程操控数据，自动判断何时重排物体以匹配训练分布
- 在未见初始状态下，使任务成功率提升至87%，优于基线方法
- 适合需要高泛化能力的机器人抓取与环境交互任务
模仿学习在多种操作任务中表现优异，但在分布外观测下（如目标物位置异常或被遮挡）常失效。此时需大量示范才能达到稳定性能。而人类面对此类异常初始状态时，会主动调整环境——如旋转杯子便于抓握、移开障碍物。本文提出ReSET算法，使机器人能自主重排初始状态，使场景更接近训练数据分布。理论上，该两阶段策略可缩小泛化差距；实践中，结合无动作依赖的人类视频与无任务依赖的远程操控数据，实现：①判断是否需重排环境，②预测人类简化动作，③映射为机器人可执行动作。对比扩散策略、视觉语言动作模型等基线，使用ReSET准备环境后，在相同训练数据量下任务成功率更高，验证了其鲁棒性优势。
原文摘要 · Abstract (English)
Imitation learning (IL) has proven effective across a wide range of manipulation tasks. However, IL policies often struggle when faced with out-of-distribution observations; for instance, when the target object is in a previously unseen position or occluded by other objects. In these cases, extensive demonstrations are needed for current IL methods to reach robust and generalizable behaviors. But when humans are faced with these sorts of atypical initial states, we often rearrange the environment for more favorable task execution. For example, a person might rotate a coffee cup so that it is easier to grasp the handle, or push a box out of the way so they can directly grasp their target object. In this work we seek to equip robot learners with the same capability: enabling robots to prepare the environment before executing their given policy. We propose ReSET, an algorithm that takes initial states -- which are outside the policy's distribution -- and autonomously modifies object poses so that the restructured scene is similar to training data. Theoretically, we show that this two step process (rearranging the environment before rolling out the given policy) reduces the generalization gap. Practically, our ReSET algorithm combines action-agnostic human videos with task-agnostic teleoperation data to i) decide when to modify the scene, ii) predict what simplifying actions a human would take, and iii) map those predictions into robot action primitives. Comparisons with diffusion policies, VLAs, and other baselines show that using ReSET to prepare the environment enables more robust task execution with equal amounts of total training data. See videos at our project website: https://reset2025paper.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。