用神经辐射场生成逼真新场景，提升机器人泛化能力
NeRF-Aug: Data Augmentation for Robotics with Neural Radiance Fields
- 用NeRF生成未见物体的逼真3D场景用于数据增强
- 在9个新物体上平均性能提升55.6%
- 生成速度比现有方法快63%，适合机器人视觉训练
训练能泛化到未知物体的机器人策略是机器人领域长期挑战。当场景中出现训练时未见过的物体时，策略性能常大幅下降。为此，我们提出NeRF-Aug，一种利用神经辐射场生成逼真3D场景以增强数据的新方法。该方法兼具生成速度快、图像真实感强和三维一致性优点。相比现有方法，生成速度提升63%，同时生成更逼真的数据。我们在5项任务中验证了其有效性，针对9个未出现在专家示范中的新物体，相比次优方法平均性能提升55.6%。视频演示见https://nerf-aug.github.io。
原文摘要 · Abstract (English)
Training a policy that can generalize to unknown objects is a long standing challenge within the field of robotics. The performance of a policy often drops significantly in situations where an object in the scene was not seen during training. To solve this problem, we present NeRF-Aug, a novel method that is capable of teaching a policy to interact with objects that are not present in the dataset. This approach differs from existing approaches by leveraging the speed, photorealism, and 3D consistency of a neural radiance field for augmentation. NeRF-Aug both creates more photorealistic data and runs 63% faster than existing methods. We demonstrate the effectiveness of our method on 5 tasks with 9 novel objects that are not present in the expert demonstrations. We achieve an average performance boost of 55.6% when comparing our method to the next best method. You can see video results at https://nerf-aug.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。