让机器人在真实世界中稳定执行远程操控,解决仿真与现实的差距问题。
MOSAIC: Bridging the Sim-to-Real Gap in Generalist Humanoid Motion Tracking and Teleoperation with Rapid Residual Adaptation
- 用强化学习训练通用动作追踪器,强调世界坐标一致性。
- 仅需少量真实数据快速适应不同操控接口,性能超越微调。
- 支持长时间远程操控,适合真实场景中的复杂任务部署。
通用类人机器人动作追踪系统近年来在仿真环境中表现优异,但在长时间远程操控中常因接口与动力学误差而失效。本文提出MOSAIC,一个开源全栈系统,支持多接口下的人类动作追踪与全身远程操控。MOSAIC首先通过强化学习,在多源动作库上训练一个面向远程操控的通用动作追踪器,采用自适应重采样和强调世界帧运动一致性的奖励机制,这对移动远程操控至关重要。为弥合仿真到现实的接口差距而不牺牲通用性,MOSAIC引入快速残差适配:仅用少量特定接口数据训练接口专用策略,并通过加性残差模块将其蒸馏至通用追踪器,性能优于直接微调或持续学习。我们在系统性消融实验、分布外基准测试及真实机器人实验中验证了MOSAIC,结果表明其在现实延迟与噪声条件下仍能实现稳健的离线动作回放与在线长时程远程操控。项目主页:baai-humanoid.github.io/MOSAIC。
原文摘要 · Abstract (English)
Generalist humanoid motion trackers have recently achieved strong simulation metrics by scaling data and training, yet often remain brittle on hardware during sustained teleoperation due to interface- and dynamics-induced errors. We present MOSAIC, an open-source, full-stack system for humanoid motion tracking and whole-body teleoperation across multiple interfaces. MOSAIC first learns a teleoperation-oriented general motion tracker via RL on a multi-source motion bank with adaptive resampling and rewards that emphasize world-frame motion consistency, which is critical for mobile teleoperation. To bridge the sim-to-real interface gap without sacrificing generality, MOSAIC then performs rapid residual adaptation: an interface-specific policy is trained using minimal interface-specific data, and then distilled into the general tracker through an additive residual module, outperforming naive fine-tuning or continual learning. We validate MOSAIC with systematic ablations, out-of-distribution benchmarking, and real-robot experiments demonstrating robust offline motion replay and online long-horizon teleoperation under realistic latency and noise. Project page: baai-humanoid.github.io/MOSAIC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。