用真实机器人数据训练的视频世界模拟器,支持闭环策略学习与真实效果验证。
GE-Sim 2.0: A Roadmap Towards Comprehensive Closed-loop Video World Simulators for Robotic Manipulation

- 基于动作条件视频生成,融合上千小时真实机器人操作数据。
- 25帧模拟仅需2.3秒,20亿参数模型在公开榜单领先。
- 可自动评估任务成功并指导策略训练,适合机器人研发人员使用。
我们提出GE-Sim 2.0(Genie Envisioner世界模拟器2.0),一种用于机器人操作的闭环视频世界模拟器。在原始动作条件视频生成框架基础上,基于数千小时真实机器人数据(包含远程操控、高接触交互及机载策略部署)重新训练,显著提升动作跟随精度与轨迹覆盖范围。在此基础上,新增三个模块实现闭环:状态专家从视频潜在表示中解码本体感知状态,支持下游视觉-语言-动作(VLA)策略的下一阶段预测;世界裁判模块根据任务指令评分生成的轨迹,提供机器可验证的成功信号与奖励,替代人工评估;加速框架可在单张H100上实现25帧滚动输出仅需2.3秒,并支持推理时最多4倍帧跳过,适用于长时序评估。GE-Sim 2.0以仅20亿参数在公开WorldArena榜单中领先,优于专用机器人世界模型与闭源通用视频生成器,且基于其生成数据训练的策略在真实世界中表现提升,确立了其作为可扩展评估与闭环学习平台的实用性。
原文摘要 · Abstract (English)
We introduce GE-Sim 2.0 (Genie Envisioner World Simulator 2.0), a closed-loop video world simulator for robotic manipulation. Building on the action-conditioned video generation framework of Genie Envisioner, GE-Sim 2.0 is re-trained on thousands of hours of real-world robot data spanning teleoperation, contact-rich interaction, and on-robot policy deployment, substantially improving action-following fidelity and trajectory coverage. On top of this foundation, three new modules close the loop from video simulation to policy learning: a state expert that decodes proprioceptive state from video latents to support next-chunk prediction by downstream VLA policies; a world judge that scores generated rollouts against task instructions, yielding machine-verifiable success signals and rewards in place of manual inspection; and an acceleration framework that delivers a 25-frame rollout in 2.3 seconds on a single H100, with up to 4* frame skipping at inference for long-horizon evaluation. GE-Sim 2.0 tops the public WorldArena leaderboard at only 2B parameters, outperforming both dedicated robotic world models and closed-source general video generators, and policies trained against its rollouts and rewards translate into measurable real-world gains, establishing GE-Sim 2.0 as a practical platform for scalable evaluation and closed-loop learning of manipulation policies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。