用包围盒引导模拟数据生成，让机器人零样本迁移到真实世界操作。
ManiBox: Enhancing Embodied Spatial Generalization via Scalable Simulation Data Generations
- 用强化学习教师生成海量模拟数据，学生策略只看物体包围盒。
- 在真实场景中成功实现零样本迁移，任务成功率随数据量呈饱和增长。
- 发现空间泛化能力与数据量呈幂律关系，适合做机器人操作训练。
具身智能体需要强大的空间感知能力以完成精确的现实操作。然而，当前方法常难以准确定位物体空间位置。虽然大量数据有助于提升空间理解，但使用真实机器人收集成本过高，而依赖仿真数据又易导致真实部署时出现视觉泛化差距。为此，我们提出ManiBox，一种基于包围盒引导的新型框架。通过解耦感知与策略泛化，有效缩小了Sim2Real差距，利用互联网规模数据，并实现仿真策略数据的可扩展采集。具体而言，强化学习教师策略高效生成大规模仿真数据；学生策略从该数据中蒸馏而来，仅以包围盒为输入，已被证明足以确定物体空间位置，从而实现零样本迁移到真实机器人。在模拟与真实环境中的全面评估表明，ManiBox在多种操作任务与场景中展现出强空间泛化与适应性。此外，我们的实证研究初步验证了空间缩放定律：空间泛化所需数据量随空间体积呈幂律增长；在特定空间体积下，任务成功率随数据量遵循米氏动力学，呈现饱和现象。视频与代码见 https://thkkk.github.io/manibox
原文摘要 · Abstract (English)
Embodied agents require robust spatial intelligence to execute precise real-world manipulations. However, this remains a significant challenge, as current methods often struggle to accurately position objects in space. Collecting extensive data can help address this issue by enhancing the agent's spatial understanding. Nonetheless, obtaining such data with real robots is prohibitively expensive, and relying on simulation data frequently leads to visual generalization gaps during real-world deployment. To tackle these challenges, we propose ManiBox, a novel bounding-box-guided framework. By decoupling perception from policy generalization, ManiBox effectively reduces the Sim2Real gap, leverages Internet-scale data, and scales our policy data collection in simulation. Specifically, within ManiBox, the RL teacher policy efficiently generates scalable simulation data. The student policy is distilled from this data and takes bounding boxes as input, which is proven sufficient for determining objects' spatial positions, thus enabling zero-shot transfer to real robots. Comprehensive evaluations in both simulated and real-world environments demonstrate that ManiBox exhibits strong spatial generalization and adaptability across various manipulation tasks and settings. Furthermore, our empirical study provides preliminary verification of spatial scaling laws, i.e., the amount of data required for spatial generalization scales with spatial volume following a power-law relationship. At a given spatial volume level, the success rate of manipulation tasks follows Michaelis-Menten kinetics with respect to data volume, exhibiting a saturation effect as data increases. Our videos and code are available at https://thkkk.github.io/manibox
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。