无需仿真,直接生成真实3D数据,提升机器人操作的泛化能力。
R2RGEN: Real-to-Real 3D Data Generation for Spatially Generalized Manipulation
- 基于真实数据点云,三阶段流程实现无仿真数据生成
- 在多种相机视角下生成多样化空间配置数据,提升训练效率
- 适合移动操作场景,可直接用于真实机器人系统
为实现通用机器人操作,空间泛化能力至关重要,要求策略在物体、环境及机器人自身空间分布变化时仍能稳健工作。传统方法依赖大量人工示范以覆盖不同空间配置,而现有数据生成方法常面临严重仿真到现实差距,且局限于固定基座和预设相机视角。本文提出一种真实到真实3D数据生成框架R2RGen,直接扩充真实点云观测-动作对生成新数据。该方法无需仿真器与渲染,高效且即插即用。具体包括:(1)在共享3D空间中解析场景与轨迹,处理不同相机设置下的源示范;(2)采用分组回溯策略增广物体与机械臂位置;(3)通过相机感知后处理对齐生成数据与真实3D传感器分布。实验表明,R2RGen显著提升数据效率,并展现出在移动操作场景中扩展与应用的巨大潜力。
原文摘要 · Abstract (English)
Towards the aim of generalized robotic manipulation, spatial generalization is the most fundamental capability that requires the policy to work robustly under different spatial distribution of objects, environment and agent itself. To achieve this, substantial human demonstrations need to be collected to cover different spatial configurations for training a generalized visuomotor policy via imitation learning. Prior works explore a promising direction that leverages data generation to acquire abundant spatially diverse data from minimal source demonstrations. However, most approaches face significant sim-to-real gap and are often limited to constrained settings, such as fixed-base scenarios and predefined camera viewpoints. In this paper, we propose a real-to-real 3D data generation framework (R2RGen) that directly augments the pointcloud observation-action pairs to generate real-world data. R2RGen is simulator- and rendering-free, thus being efficient and plug-and-play. Specifically, we propose a unified three-stage framework, which (1) pre-processes source demonstrations under different camera setups in a shared 3D space with scene / trajectory parsing; (2) augments objects and robot's position with a group-wise backtracking strategy; (3) aligns the distribution of generated data with real-world 3D sensor using camera-aware post-processing. Empirically, R2RGen substantially enhances data efficiency on extensive experiments and demonstrates strong potential for scaling and application on mobile manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。