用多视角提升手持机械臂数据采集效果，让机器人学得更广。
MV-UMI: A Scalable Multi-View Interface for Cross-Embodiment Learning
- 融合第一人称与第三人称视角，增强场景理解
- 在3个任务中提升47%的子任务性能
- 适合想用便携设备做跨机器人学习的研究者
模仿学习在机器人操作政策开发中展现出巨大潜力，但依赖多样且高质量的数据集，而这类数据收集成本高、受限于特定机器人形态。便携式手持夹持器作为新型数据采集方式，虽具可扩展性，但仅依赖腕部第一人称摄像头，难以捕捉充分场景上下文。本文提出MV-UMI（多视角通用操作接口），将第三人称视角与第一人称摄像头结合，缓解人类示范与机器人部署间的领域差异，保留手持设备的跨形态优势。实验结果（含消融研究）表明，该框架在需广泛场景理解的子任务中性能平均提升约47%，验证了其在不牺牲跨形态优势的前提下，显著拓展了手持夹持系统可学习操作任务的范围。
原文摘要 · Abstract (English)
Recent advances in imitation learning have shown great promise for developing robust robot manipulation policies from demonstrations. However, this promise is contingent on the availability of diverse, high-quality datasets, which are not only challenging and costly to collect but are often constrained to a specific robot embodiment. Portable handheld grippers have recently emerged as intuitive and scalable alternatives to traditional robotic teleoperation methods for data collection. However, their reliance solely on first-person view wrist-mounted cameras often creates limitations in capturing sufficient scene contexts. In this paper, we present MV-UMI (Multi-View Universal Manipulation Interface), a framework that integrates a third-person perspective with the egocentric camera to overcome this limitation. This integration mitigates domain shifts between human demonstration and robot deployment, preserving the cross-embodiment advantages of handheld data-collection devices. Our experimental results, including an ablation study, demonstrate that our MV-UMI framework improves performance in sub-tasks requiring broad scene understanding by approximately 47% across 3 tasks, confirming the effectiveness of our approach in expanding the range of feasible manipulation tasks that can be learned using handheld gripper systems, without compromising the cross-embodiment advantages inherent to such systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。