用随机积木训练机器人，能零样本抓取真实物体。
Learning to Grasp Anything by Playing with Random Toys
- 用四种基本形状拼出随机玩具，训练机器人抓取。
- 在真实物体上实现67%抓取成功率，零样本表现优。
- 关键在物体中心视觉表征，适合做通用抓取研究。
机器人抓取策略常难以泛化到新物体，限制了实际应用。受儿童通过玩简单玩具发展通用操作能力的启发，我们探索机器人是否也能通过类似方式实现泛化。结果表明，仅用四个基础形状（球、立方体、圆柱、环）随机组合的“玩具”即可训练出具备泛化能力的抓取模型。该方法在仿真与物理机器人上均表现出色，对真实世界物体实现67%的抓取成功率，显著优于依赖大量同类数据的先进方法。关键在于我们提出的检测池化机制所生成的物体中心视觉表征。我们进一步研究了训练玩具数量与多样性、每类玩具演示次数对零样本性能的影响。项目代码、视频与数据集已公开。
原文摘要 · Abstract (English)
Robotic manipulation policies often struggle to generalize to novel objects, limiting their real-world utility. In contrast, cognitive science suggests that children develop generalizable dexterous manipulation skills by mastering a small set of simple toys and then applying that knowledge to more complex items. Inspired by this, we study if similar generalization capabilities can also be achieved by robots. Our results indicate robots can learn generalizable grasping using randomly assembled objects that are composed from just four shape primitives: spheres, cuboids, cylinders, and rings. We show that training on these "toys" enables robust generalization to real-world objects, yielding strong zero-shot performance. Crucially, we find the key to this generalization is an object-centric visual representation induced by our proposed detection pooling mechanism. Evaluated in both simulation and on physical robots, our model achieves a 67% real-world grasping success rate on the YCB dataset, outperforming state-of-the-art approaches that rely on substantially more in-domain data. We further study how zero-shot generalization performance scales by varying the number and diversity of training toys and the demonstrations per toy. We believe this work offers a promising path to scalable and generalizable learning in robotic manipulation. Demonstration videos, code, checkpoints and our dataset are available on our project page: https://lego-grasp.github.io/ .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。