用单个XR设备实现跨机器人形态的全身遥操作,无需重训
X-OP: Cross-Morphology Whole-Body Teleoperation via MPC Retargeting

- 基于MPC的运动重定向,兼顾操作者意图与机器人动态可行性
- 仿真中人形机器人任务完成时间降低30%以上,移动机械臂零碰撞
- 支持真实场景即插即用,用户可实时调整操作行为偏好
全身遥操作对本地操作任务中的机器人数据收集至关重要,但现有依赖外骨骼或多相机系统的方案成本高、复杂且受环境限制。近期使用单一扩展现实(XR)设备配合端到端强化学习策略的方法部分缓解了这些问题,但仍需针对特定机器人重训,易出现分布外失败,且运动重定向忽略动态可行性。我们提出一种分层全身遥操作框架,仅需单个XR设备,即可在不重训特定机器人策略的前提下泛化于多种机器人形态。基于模型预测控制(MPC)的运动重定向器联合优化操作者意图与机器人动态可行性,生成适配现有底层控制器的最优指令。为确保在线执行鲁棒性,引入状态同步方法,在每个MPC步骤重置仿真状态以应对噪声测量与接触敏感问题,并集成基于SLAM的全局位姿反馈以抑制长期漂移。仿真结果显示,该方法在人形机器人(完成时间降低超30%,能耗降低20%)和移动机械臂(零碰撞)任务中均显著优于基线。真实世界实验进一步验证了方法的有效性与灵活性,成功将该重定向器部署于两类平台,实现全身控制任务,并支持用户根据偏好灵活调整操作行为。该即插即用框架提供了一种可扩展、形态无关的全身机器人遥操作解决方案,支持实时行为定制与跨平台广泛应用。
原文摘要 · Abstract (English)
Whole-body teleoperation is essential for scalable robot data collection in loco-manipulation tasks, yet existing approaches relying on exoskeleton suits or multi-camera setups impose prohibitive cost, complexity, and environmental constraints. Recent methods using a single extended reality (XR) device with end-to-end reinforcement learning policies partially address these limitations but require robot-specific retraining, suffer from out-of-distribution failures, and rely on motion retargeting that neglects dynamic feasibility. We propose a hierarchical whole-body teleoperation framework driven by a single XR device that generalizes across diverse robot morphologies without retraining robot-specific policies. A Model Predictive Control (MPC)-based motion retargeter jointly optimizes alignment with the operator's intent and the robot's dynamic feasibility, generating optimal commands for existing low-level controllers. To ensure robust online execution, we introduce a state synchronization method that resets the simulator state at each MPC step to handle noisy real-world measurements and contact sensitivity, and integrate SLAM-based global pose feedback to mitigate long-term drift. Simulation results show higher success rates on whole-body control tasks for both a humanoid (over 30% lower completion time and 20% lower power consumption) and a mobile manipulator (zero collisions) compared to baselines. Real-world experiments further validate the effectiveness and flexibility of our method, demonstrating the successful deployment of the proposed retargeter on both platforms for whole-body control tasks and the ease of allowing users to adjust teleoperation behavior based on their preferences. This plug-and-play framework offers a scalable, morphology-agnostic solution for whole-body robot teleoperation, enabling real-time behavioral customization and broad applicability across platforms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。