arXiv:2606.02577cs.ROcs.CV2026-06

用虚拟世界模型生成逼真机器人数据,省去真实操作成本。

RoboDream: Compositional World Models for Scalable Robot Data Synthesis

论文配图:RoboDream: Compositional World Models for Scalable Robot Data Synthesis
图 1 · 摘自论文原文
  • 以机器人运动为锚点,结合场景与物体先验生成新环境和视角。
  • 生成数据可使下游策略性能提升,减少90%以上真实数据需求。
  • 适合需要大量多样数据的机器人学习研究者,尤其关注低成本训练。

扩大机器人学习需大量多样化示范,但通过远程操控收集真实数据仍成本高昂且耗时。尽管视频扩散模型为数据扩展提供可能,现有生成方法多局限于表面视觉增强,或产生不符合物理规律的动作幻觉。本文提出一种可泛化的以实体为中心的世界模型,通过合成包含新物体、新场景和新视角的逼真示范,实现大规模数据生成。该方法将生成过程锚定在渲染的机器人运动上,并基于显式的场景与物体先验进行条件控制,有效分离轨迹执行与环境合成。此框架具备两项强大能力:(1) 检索与重生,可将已有轨迹复用于全新情境,无需新增动作数据;(2) 无道具遥控,操作者仅在空环境中操控,模型随后幻化出目标物体与场景,消除重置时间。我们在真实任务中验证,生成数据持续提升下游策略表现,并显著降低多种操作任务的真实数据需求。

原文摘要 · Abstract (English)

Scaling robot learning requires large-scale, diverse demonstrations, yet real-world data collection via teleoperation remains prohibitively expensive and time-consuming. While video diffusion models offer a promising avenue for data scaling, existing generative approaches are often limited to superficial visual augmentation, or suffer from embodiment hallucinations that yield physically infeasible motions. We present a generalizable embodiment-centric world model that achieves scalable data generation by synthesizing photorealistic demonstrations with novel objects, in novel scenes, and from novel viewpoints. Our approach anchors generation to rendered robot motion while conditioning on explicit scene and object priors, effectively decoupling trajectory execution from environment synthesis. This formulation has the potential to unlock two powerful data scaling capabilities: (1) retrieval and rebirth, which repurposes existing trajectories into entirely new contexts without new motion data; and (2) prop-free teleoperation, where operators manipulate empty air and the model hallucinates the target objects and scene afterwards, eliminating reset time. We demonstrate with real-world experiments that our generated data consistently improves downstream policy performance and significantly reduces real-world data requirements across diverse manipulation tasks.

机器人学习数据生成世界模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。