用动态数字孪生实现仿真与真实机器人无缝切换,解决策略部署难题。
Real-is-Sim: Bridging the Sim-to-Real Gap with a Dynamic Digital Twin
- 构建60Hz同步的数字孪生,让策略始终在仿真中运行
- 虚拟评估结果与真实表现一致,验证框架有效性
- 适合需要安全试错的机器人长程任务研发
我们提出 real-is-sim,一种将仿真融入行为克隆流程的新方法。与仅依赖真实世界的方法相比,该方法可在部署前安全测试策略;与传统 sim-to-real 方法相比,无需复杂适应即可跨越仿真到真实世界的差距。其核心是一个由 Embodied Gaussian 模拟器驱动的动态数字孪生,以60Hz频率与现实世界同步。策略始终基于模拟器状态表示进行训练,并在仿真环境中执行,从不直接作用于真实机器人。部署时,真实机器人仅跟随模拟机器人的关节状态,同时通过真实测量持续校正模拟环境。这种由模拟器驱动且实时同步的架构,将跨域挑战交由数字孪生的同步机制处理,而非策略本身。我们在长程操作任务 PushT 上验证了该框架,显示虚拟评估结果与真实表现高度一致。此外,我们展示了如何用虚拟轨迹增强真实数据,并对比了基于物体位姿、静态及机器人搭载摄像头渲染图像等不同表示的学习效果。结果表明,real-is-sim 在训练、评估与部署阶段均具有高度灵活性。
原文摘要 · Abstract (English)
We introduce real-is-sim, a new approach to integrating simulation into behavior cloning pipelines. In contrast to real-only methods, which lack the ability to safely test policies before deployment, and sim-to-real methods, which require complex adaptation to cross the sim-to-real gap, our framework allows policies to seamlessly switch between running on real hardware and running in parallelized virtual environments. At the center of real-is-sim is a dynamic digital twin, powered by the Embodied Gaussian simulator, that synchronizes with the real world at 60Hz. This twin acts as a mediator between the behavior cloning policy and the real robot. Policies are trained using representations derived from simulator states and always act on the simulated robot, never the real one. During deployment, the real robot simply follows the simulated robot's joint states, and the simulation is continuously corrected with real world measurements. This setup, where the simulator drives all policy execution and maintains real-time synchronization with the physical world, shifts the responsibility of crossing the sim-to-real gap to the digital twin's synchronization mechanisms, instead of the policy itself. We demonstrate real-is-sim on a long-horizon manipulation task (PushT), showing that virtual evaluations are consistent with real-world results. We further show how real-world data can be augmented with virtual rollouts and compare to policies trained on different representations derived from the simulator state including object poses and rendered images from both static and robot-mounted cameras. Our results highlight the flexibility of the real-is-sim framework across training, evaluation, and deployment stages. Videos available at https://real-is-sim.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。