arXiv:2605.20894cs.RO2026-05被引 1

解决移动机器人模仿学习中动作污染与执行延迟问题,实现无需硬件的高效演示。

Mobile UMI: Cross-View Diffusion Policy with Decoupled Kinematics for Mobile Manipulation

论文配图:Mobile UMI: Cross-View Diffusion Policy with Decoupled Kinematics for Mobile Manipulation
图 1 · 摘自论文原文
  • 双摄像头捕捉全身与手部视角,解耦机械臂与底盘运动轨迹。
  • 通过空间锚点对齐视觉惯性帧,提取分离的三维操作与二维导航路径。
  • 异步滚动规划器实时校正状态,消除执行延迟导致的误差。

便携式演示界面下的移动模仿学习面临两大瓶颈:运动导致的动作标签污染,以及连续移动基座带来的推理延迟。近期腕戴式接口降低了桌面数据采集成本,但单一腕部视角无法捕捉全局导航上下文;增加体佩戴摄像头又使行走与手部动作纠缠。生成式策略引入数百毫秒推理延迟,导致基座在延迟期间越过预测路径点,迫使动作拼接处进行后向修正。本文提出 Mobile UMI,一种无需硬件的演示框架,通过三个组件解决上述问题:首先,双摄像头系统记录以胸部为中心的全局上下文和以手腕为中心的局部交互,无需机器人在场;其次,基于 ChArUco 的单次空间锚点统一胸与手的视觉-惯性坐标系,将手部姿态重表达为相对于胸部的坐标,从而提取解耦的 SE(3) 操控轨迹与 SE(2) 底盘轨迹;第三,异步滚动规划器执行在线状态匹配:每个生成的动作块均与当前物理姿态对齐,过期路径点在执行前被丢弃。系统在四个长时序家庭任务上评估,每任务100次试验下平均成功率达83.8%。对照实验表明,仅使用胸部相对标签即可缩小大部分差距,而在线状态匹配弥补剩余差异。结果表明,在测试条件下,显式运动学解耦结合状态级延迟对齐,可在不改变底层策略结构的前提下有效提升性能。

原文摘要 · Abstract (English)

Mobile imitation learning on portable demonstration interfaces faces two coupled bottlenecks: locomotion-contaminated action labels and inference-induced execution latency on a continuously moving base. Recent wrist-mounted interfaces lower the cost of tabletop data collection, yet a single wrist view does not capture the global context required for base navigation. Adding a body-mounted camera entangles human walking with hand motion. Meanwhile, generative policies introduce hundreds of milliseconds of inference latency, during which the base advances past predicted waypoints, forcing backward corrections at action splices. This paper presents Mobile UMI, a hardware-free demonstration framework that addresses both gaps through three components. First, a dual-camera capture system records chest-centric global context and wrist-centric local interaction without any robot present. Second, a one-shot ChArUco-based spatial anchor unifies the chest and hand visual-inertial frames; the hand pose is then re-expressed relative to the chest to extract decoupled SE(3) manipulation and SE(2) base trajectories. Third, an asynchronous receding-horizon executor performs online state matching: each generated action chunk is realigned with the current physical pose so that expired waypoints are discarded before execution. The full system is evaluated on four long-horizon household tasks, achieving an average success rate of 83.8% over 100 trials per task. Controlled comparisons against ACT and Diffusion Policy show that the chest-relative label alone closes much of the gap; online state matching closes the remainder. These results indicate that, for mobile imitation learning under the tested conditions, explicit kinematic factorization combined with state-level latency alignment provides an effective solution without requiring architectural changes to the underlying policy class.

移动操作模仿学习扩散模型动作解耦

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。