通过视觉提示编辑实现零样本场景迁移，让机器人快速适应新环境。
Robotic Scene Cloning:Advancing Zero-Shot Robotic Scene Adaptation in Manipulation via Visual Prompt Editing
- 用视觉提示编辑已有操作轨迹，实现场景自适应。
- 在模拟与真实环境中均显著提升策略泛化能力。
- 适合需要快速部署的现实机器人应用场景。
现代机器人可在训练良好的环境中完成多种简单任务并适应多样场景，但将其部署到真实用户场景时仍面临挑战，主要因其零样本能力有限，常需大量现场数据采集。为解决此问题，我们提出机器人场景克隆（RSC），一种通过编辑现有机器人操作轨迹实现场景特异性适应的新方法。RSC利用视觉提示机制和精心设计的条件注入模块，实现准确且场景一致的样本生成。不仅能迁移纹理，还能根据视觉提示进行适度的形状调整，从而在多种物体类型上表现出可靠的任务性能。在多个模拟与真实世界环境中的实验表明，RSC显著提升了目标环境中的策略泛化能力。
原文摘要 · Abstract (English)
Modern robots can perform a wide range of simple tasks and adapt to diverse scenarios in the well-trained environment. However, deploying pre-trained robot models in real-world user scenarios remains challenging due to their limited zero-shot capabilities, often necessitating extensive on-site data collection. To address this issue, we propose Robotic Scene Cloning (RSC), a novel method designed for scene-specific adaptation by editing existing robot operation trajectories. RSC achieves accurate and scene-consistent sample generation by leveraging a visual prompting mechanism and a carefully tuned condition injection module. Not only transferring textures but also performing moderate shape adaptations in response to the visual prompts, RSC demonstrates reliable task performance across a variety of object types. Experiments across various simulated and real-world environments demonstrate that RSC significantly enhances policy generalization in target environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。