用低成本设备生成3D动作数据，让机器人跨平台快速学习操作。
UMIGen: A Unified Framework for Egocentric Point Cloud Generation and Cross-Embodiment Robotic Imitation Learning
- 手持设备同步采集点云与动作，无需视觉定位系统。
- 只生成相机视场内的点云，提升数据真实性和效率。
- 数据可直接跨机器人使用，加速真实世界任务学习。
基于数据的机器人学习面临困境：可靠策略需要大规模高质量示范数据，但收集成本高、依赖专用硬件，且现有方法空间泛化能力有限。通用操作接口（UMI）虽降低了硬件要求，但仅能捕捉场景的RGB图像，忽略了任务依赖的3D几何信息。受DemoGen启发，我们提出UMIGen统一框架，包含两个核心组件：(1) Cloud-UMI，一种无需视觉SLAM的手持数据采集设备，可同步记录点云观测-动作对；(2) 可见性感知优化机制，通过仅生成相机视场内的点云，将DemoGen流程扩展至以第一人称视角的3D观测。该框架实现与真实第一人称观测一致的高效数据生成，且可直接跨不同机器人形态迁移，无需后处理。仿真与真实环境实验表明，UMIGen支持强跨体感泛化，并显著加速多样化操作任务的数据收集。
原文摘要 · Abstract (English)
Data-driven robotic learning faces an obvious dilemma: robust policies demand large-scale, high-quality demonstration data, yet collecting such data remains a major challenge owing to high operational costs, dependence on specialized hardware, and the limited spatial generalization capability of current methods. The Universal Manipulation Interface (UMI) relaxes the strict hardware requirements for data collection, but it is restricted to capturing only RGB images of a scene and omits the 3D geometric information on which many tasks rely. Inspired by DemoGen, we propose UMIGen, a unified framework that consists of two key components: (1) Cloud-UMI, a handheld data collection device that requires no visual SLAM and simultaneously records point cloud observation-action pairs; and (2) a visibility-aware optimization mechanism that extends the DemoGen pipeline to egocentric 3D observations by generating only points within the camera's field of view. These two components enable efficient data generation that aligns with real egocentric observations and can be directly transferred across different robot embodiments without any post-processing. Experiments in both simulated and real-world settings demonstrate that UMIGen supports strong cross-embodiment generalization and accelerates data collection in diverse manipulation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。