用仿真回放验证动作质量，提升机器人合成数据可靠性
RoboCurate: Harnessing Diversity with Action-Verified Neural Trajectory for Robot Learning
- 通过仿真回放对比视频动作与真实运动一致性来筛选高质量动作
- 在真实世界任务中成功率提升最高达179.9%
- 适合需要高精度动作数据的机器人学习研究者
由视频生成模型合成的仿真数据在机器人学习中展现出可扩展性优势，但常因视频生成不准确导致动作质量不一致。尽管视觉语言模型（VLMs）可用于评估视频质量，却难以区分物理合理性，更无法直接评价生成动作本身。为此，我们提出RoboCurate，一种新型合成机器人数据生成框架，通过将预测动作在仿真器中回放，并比较仿真轨迹与生成视频之间的运动一致性来评估动作质量。此外，利用图像到图像编辑拓展观测多样性，并通过保持动作不变的视频到视频迁移进一步丰富外观表现。实验表明，使用RoboCurate生成的数据相比仅用真实数据，在GR-1 Tabletop（300次演示）上成功率提升70.1%，在DexMimicGen预训练设置下提升16.1%，在挑战性的ALLEX人形灵巧操作真实场景中提升高达179.9%。
原文摘要 · Abstract (English)
Synthetic data generated by video generative models has shown promise for robot learning as a scalable pipeline, but it often suffers from inconsistent action quality due to imperfectly generated videos. Recently, vision-language models (VLMs) have been leveraged to validate video quality, but they have limitations in distinguishing physically accurate videos and, even then, cannot directly evaluate the generated actions themselves. To tackle this issue, we introduce RoboCurate, a novel synthetic robot data generation framework that evaluates and filters the quality of annotated actions by comparing them with simulation replay. Specifically, RoboCurate replays the predicted actions in a simulator and assesses action quality by measuring the consistency of motion between the simulator rollout and the generated video. In addition, we unlock observation diversity beyond the available dataset via image-to-image editing and apply action-preserving video-to-video transfer to further augment appearance. We observe RoboCurate's generated data yield substantial relative improvements in success rates compared to using real data only, achieving +70.1% on GR-1 Tabletop (300 demos), +16.1% on DexMimicGen in the pre-training setup, and +179.9% in the challenging real-world ALLEX humanoid dexterous manipulation setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。