用3D生成模型扩增单次示范，让机器人从任意角度操作物体。
Learning in ImaginationLand: Omnidirectional Policies through 3D Generative Models (OP-Gen)
- 用3D生成模型构建虚拟数据集，扩展单次真实演示。
- 机器人可在物体对面起始完成任务，减少示范次数。
- 适合需要少样本学习的机器人操控场景。
近期的3D生成模型能够仅凭几张图像生成完整物体形状，为机器人学带来新机遇。本文表明，这些3D生成模型可用于扩增单次真实示范的数据集，从而在该虚构数据集中学习全向策略（Omnidirectional Policy）。实验发现，此方法使机器人能在与示范状态相距甚远的情况下完成任务，包括从物体另一侧起始，显著降低对示范数量的需求。我们在抓取、开抽屉、投掷垃圾等真实任务中验证了该策略，通过分析不同设计选择对策略行为的影响，结果优于使用其他数据增强方法的最新基线。
原文摘要 · Abstract (English)
Recent 3D generative models, which are capable of generating full object shapes from just a few images, now open up new opportunities in robotics. In this work, we show that 3D generative models can be used to augment a dataset from a single real-world demonstration, after which an omnidirectional policy can be learned within this imagined dataset. We found that this enables a robot to perform a task when initialised from states very far from those observed during the demonstration, including starting from the opposite side of the object relative to the real-world demonstration, significantly reducing the number of demonstrations required for policy learning. Through several real-world experiments across tasks such as grasping objects, opening a drawer, and placing trash into a bin, we study these omnidirectional policies by investigating the effect of various design choices on policy behaviour, and we show superior performance to recent baselines which use alternative methods for data augmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。