仅用一段视频，让四足机器人自动学会走路技能。
SDS -- See it, Do it, Sorted: Quadruped Skill Synthesis from Single Video Demonstration
- 用视觉编码与结构分解技术，从视频生成可执行的奖励函数。
- 在仿真和真实世界中实现100%步态匹配，动态时间规整距离达10^-6。
- 无需标签或人工调奖，适合快速部署到不同体型四足机器人。
想象一个机器人仅通过观看任意一段视频就能学会行走技能，无需标签或奖励工程。我们提出SDS（“看它，做它，已排序”），一种从非结构化示范中自动获取技能的流水线。利用GPT-4o，SDS采用新颖的提示技术——时空网格视觉编码（$G_{v}$）与结构化输入分解（SUS），将原始视频转化为可执行的奖励函数（RF）。这些奖励函数用于训练PPO策略，并通过闭环进化优化，以训练视频和性能指标作为自监督信号。SDS使四足机器人（如Unitree Go1）学会四种步态——走、跳、小跑、跃动，在仿真和真实世界中均达到100%步态匹配精度，动态时间规整（DTW）距离约为$10^{-6}$，且无失败地稳定行走。该方法可泛化至不同形态的四足机器人（如ANYmal），在数据效率、训练时间与工程成本上优于现有工作。更多资料与代码开源：https://rpl-cs-ucl.github.io/SDSweb/
原文摘要 · Abstract (English)
Imagine a robot learning locomotion skills from any single video, without labels or reward engineering. We introduce SDS ("See it. Do it. Sorted."), an automated pipeline for skill acquisition from unstructured demonstrations. Using GPT-4o, SDS applies novel prompting techniques, in the form of spatio-temporal grid-based visual encoding ($G_{v}$) and structured input decomposition (SUS). These produce executable reward functions (RF) from the raw input videos. The RFs are used to train PPO policies and are optimized through closed-loop evolution, using training footage and performance metrics as self-supervised signals. SDS allows quadrupeds (e.g. Unitree Go1) to learn four gaits -- trot, bound, pace, and hop -- achieving 100% gait matching fidelity, Dynamic Time Warping (DTW) distance in the order of $10^{-6}$, and stable locomotion with zero failures, both in simulation and the real world. SDS generalizes to morphologically different quadrupeds (e.g. ANYmal) and outperforms prior work in data efficiency, training time and engineering effort. Further materials and the code are open-source under: https://rpl-cs-ucl.github.io/SDSweb/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。