用生成模型合成真实感视觉数据，让机器人狗在仿真中学会视觉障碍跑酷。
Learning Visual Parkour from Generated Images
- 用生成模型从机器人视角合成多样且物理准确的图像序列。
- 零样本迁移至真实机器人，仅用普通彩色摄像头实现成功跑酷。
- 适合对仿真训练、视觉导航感兴趣的科研与工程人员。
快速精确的物理模拟是机器人学习的关键，使机器人能在现实中难以复现的失败场景中探索，并利用无限的策略内数据进行学习。然而，如何将丰富真实的RGB感知融入仿真到现实的迁移流程仍具挑战。本文训练机器人狗在仿真环境中完成视觉障碍跑酷任务。我们提出一种方法，利用生成模型从机器人自身视角合成多样且物理准确的场景图像序列。实验展示了该方法在仅使用低成本通用彩色摄像头的实时视觉观测下，实现零样本迁移至真实机器人并成功执行动作。更多详情请访问：https://lucidsim.github.io
原文摘要 · Abstract (English)
Fast and accurate physics simulation is an essential component of robot learning, where robots can explore failure scenarios that are difficult to produce in the real world and learn from unlimited on-policy data. Yet, it remains challenging to incorporate RGB-color perception into the sim-to-real pipeline that matches the real world in its richness and realism. In this work, we train a robot dog in simulation for visual parkour. We propose a way to use generative models to synthesize diverse and physically accurate image sequences of the scene from the robot's ego-centric perspective. We present demonstrations of zero-shot transfer to the RGB-only observations of the real world on a robot equipped with a low-cost, off-the-shelf color camera. website visit https://lucidsim.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。