用虚拟现实生成逼真机器人数据，训练出能直接上手真实环境的视觉策略。
Lucid-XR: An Extended-Reality Data Engine for Robotic Manipulation
- 基于网页版物理仿真，头戴设备即可实时交互生成数据
- 合成数据训练后零样本迁移至杂乱昏暗的真实场景
- 支持软材料、松散颗粒等复杂操作任务，适合具身智能研究
我们提出Lucid-XR，一个用于训练真实机器人系统的生成式数据引擎。其核心是vuer——一个运行在XR头显上的基于Web的物理仿真环境，无需专用设备即可实现无延迟的沉浸式交互，支持互联网规模访问。整个系统整合了本地物理仿真与人机动作迁移技术，通过自然语言控制的物理引导视频生成流程进一步扩增数据。实验表明，仅使用Lucid-XR合成数据训练的机器人视觉策略，可零样本迁移到未见过的杂乱、昏暗环境中。案例涵盖涉及软性材料、松散颗粒和刚体接触的灵巧操作任务。项目主页：https://lucidxr.github.io
原文摘要 · Abstract (English)
We introduce Lucid-XR, a generative data engine for creating diverse and realistic-looking multi-modal data to train real-world robotic systems. At the core of Lucid-XR is vuer, a web-based physics simulation environment that runs directly on the XR headset, enabling internet-scale access to immersive, latency-free virtual interactions without requiring specialized equipment. The complete system integrates on-device physics simulation with human-to-robot pose retargeting. Data collected is further amplified by a physics-guided video generation pipeline steerable via natural language specifications. We demonstrate zero-shot transfer of robot visual policies to unseen, cluttered, and badly lit evaluation environments, after training entirely on Lucid-XR's synthetic data. We include examples across dexterous manipulation tasks that involve soft materials, loosely bound particles, and rigid body contact. Project website: https://lucidxr.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。