用人体外骨骼捕捉真实交互数据,实现高精度模拟与现实间的无缝迁移。
ExoGS: A 4D Real-to-Sim-to-Real Framework for Scalable Manipulation Data Collection
- 通过自研外骨骼设备捕捉人操作时的动态轨迹与视觉数据,保证动作一致性。
- 在仿真中重建可编辑的3D高斯点云资产,支持大规模数据增强和几何一致回放。
- 适用于需要大量高质量交互数据的机器人操控任务,尤其适合接触密集型操作。
真实到模拟再到真实的技术在机器人操控领域日益受到关注,因其能在仿真中生成可扩展的数据并缩小仿真与现实之间的差距。然而,以往方法主要聚焦于环境层面的视觉真实到模拟转移,忽略了交互过程的传递,而这类信息在纯仿真中获取困难且效率低下,尤其是在接触密集型任务中。本文提出 ExoGS,一种无需机器人的4D真实到模拟再到真实框架,能够同时捕获真实世界中的静态环境与动态交互,并无缝转移到仿真环境。ExoGS 使用自研的类机器人被动外骨骼 AirExo-3,以毫米级精度同步采集运动轨迹与RGB图像,用于人类直接示范。机器人、物体与环境被重建为可编辑的3D高斯点云资产,实现几何一致的重放与大规模数据增强。此外,引入轻量级掩码适配器,在策略中注入实例级语义,提升在视觉域偏移下的鲁棒性。真实实验表明,相比基于遥操作的基线方法,ExoGS 显著提升了数据效率与策略泛化能力。代码与硬件文件已公开于 https://github.com/zaixiabalala/ExoGS。
原文摘要 · Abstract (English)
Real-to-Sim-to-Real technique is gaining increasing interest for robotic manipulation, as it can generate scalable data in simulation while having narrower sim-to-real gap. However, previous methods mainly focused on environment-level visual real-to-sim transfer, ignoring the transfer of interactions, which could be challenging and inefficient to obtain purely in simulation especially for contact-rich tasks. We propose ExoGS, a robot-free 4D Real-to-Sim-to-Real framework that captures both static environments and dynamic interactions in the real world and transfers them seamlessly to a simulated environment. It provides a new solution for scalable manipulation data collection and policy learning. ExoGS employs a self-designed robot-isomorphic passive exoskeleton AirExo-3 to capture kinematically consistent trajectories with millimeter-level accuracy and synchronized RGB observations during direct human demonstrations. The robot, objects, and environment are reconstructed as editable 3D Gaussian Splatting assets, enabling geometry-consistent replay and large-scale data augmentation. Additionally, a lightweight Mask Adapter injects instance-level semantics into the policy to enhance robustness under visual domain shifts. Real-world experiments demonstrate that ExoGS significantly improves data efficiency and policy generalization compared to teleoperation-based baselines. Code and hardware files have been released on https://github.com/zaixiabalala/ExoGS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。