arXiv:2603.08546cs.ROcs.CV2026-03被引 26

用快速稳定的物理模拟器生成机器人训练数据,效果接近真实世界。

Interactive World Simulator for Robot Policy Training and Evaluation

  • 用一致性模型实现图像解码与潜空间动态预测,提升模拟速度与稳定性。
  • 在单张RTX 4090上以15帧/秒运行超10分钟长时物理交互模拟。
  • 生成数据训练的策略表现媲美真实数据,适合大规模机器人训练与评估。

动作条件视频预测模型(常称世界模型)在机器人应用中潜力巨大,但现有方法通常较慢,且难以在长时程内保持物理一致性,限制了其在可扩展机器人策略训练与评估中的使用。我们提出交互式世界模拟器框架,基于中等规模机器人交互数据集构建交互世界模型。该方法利用一致性模型进行图像解码和潜空间动态预测,实现快速稳定的物理交互模拟。实验表明,所学世界模型能生成一致的像素级预测,在单张RTX 4090 GPU上以15 FPS持续稳定模拟超过10分钟的长时交互。该框架支持仅在世界模型内完成可扩展的演示收集,用于训练顶尖模仿策略。通过在包含刚性物体、柔性物体、物体堆叠及其交互的多样化任务中进行大量真实世界评估,发现基于世界模型生成数据训练的策略性能与真实数据训练相当。此外,在世界模型和真实世界中对策略进行评估,观察到模拟与真实性能之间存在强相关性。这些结果确立了交互式世界模拟器作为可扩展机器人数据生成与忠实、可复现策略评估的稳定物理一致替代方案。

原文摘要 · Abstract (English)

Action-conditioned video prediction models (often referred to as world models) have shown strong potential for robotics applications, but existing approaches are often slow and struggle to capture physically consistent interactions over long horizons, limiting their usefulness for scalable robot policy training and evaluation. We present Interactive World Simulator, a framework for building interactive world models from a moderate-sized robot interaction dataset. Our approach leverages consistency models for both image decoding and latent-space dynamics prediction, enabling fast and stable simulation of physical interactions. In our experiments, the learned world models produce interaction-consistent pixel-level predictions and support stable long-horizon interactions for more than 10 minutes at 15 FPS on a single RTX 4090 GPU. Our framework enables scalable demonstration collection solely within the world models to train state-of-the-art imitation policies. Through extensive real-world evaluation across diverse tasks involving rigid objects, deformable objects, object piles, and their interactions, we find that policies trained on world-model-generated data perform comparably to those trained on the same amount of real-world data. Additionally, we evaluate policies both within the world models and in the real world across diverse tasks, and observe a strong correlation between simulated and real-world performance. Together, these results establish the Interactive World Simulator as a stable and physically consistent surrogate for scalable robotic data generation and faithful, reproducible policy evaluation.

世界模型机器人训练物理模拟数据生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。