arXiv:2410.14790cs.CVcs.AI2024-10被引 11

机器人用自监督学习自动规划最佳视角,高效重建植物3D模型。

SSL-NBV: A Self-Supervised-Learning-Based Next-Best-View algorithm for Efficient 3D Plant Reconstruction by a Robot

  • 通过自监督学习在线生成训练数据,无需人工标注。
  • 仅需少量视角即可完成重建,比传统方法快800倍以上。
  • 支持实时适应新环境,适合动态农业场景应用。

植物3D重建因复杂形状导致大量遮挡而困难。下一代视图(NBV)方法通过迭代选择新视角以最大化信息增益(IG)。基于深度学习的NBV(DL-NBV)方法虽计算效率更高,但需依赖大量真实植物模型训练,且依赖离线数据,难以适应变化的农业环境。本文提出一种基于自监督学习的NBV方法(SSL-NBV),利用深度神经网络预测候选视角的信息增益。该方法在任务执行中自主收集训练数据,通过对比前后3D传感器数据,结合弱监督学习与经验回放实现高效在线学习。仿真与真实环境下的交叉验证表明,SSL-NBV所需视角数少于非NBV方法,且比体素法快超过800倍;相比基线DL-NBV,训练标注减少90%以上,并可通过在线微调适应新场景。使用真实植物的实验也证明,该方法能有效学习新视角规划策略。最重要的是,SSL-NBV实现了全流程自动化训练与持续在线学习,可在动态农业环境中运行。

原文摘要 · Abstract (English)

The 3D reconstruction of plants is challenging due to their complex shape causing many occlusions. Next-Best-View (NBV) methods address this by iteratively selecting new viewpoints to maximize information gain (IG). Deep-learning-based NBV (DL-NBV) methods demonstrate higher computational efficiency over classic voxel-based NBV approaches but current methods require extensive training using ground-truth plant models, making them impractical for real-world plants. These methods, moreover, rely on offline training with pre-collected data, limiting adaptability in changing agricultural environments. This paper proposes a self-supervised learning-based NBV method (SSL-NBV) that uses a deep neural network to predict the IG for candidate viewpoints. The method allows the robot to gather its own training data during task execution by comparing new 3D sensor data to the earlier gathered data and by employing weakly-supervised learning and experience replay for efficient online learning. Comprehensive evaluations were conducted in simulation and real-world environments using cross-validation. The results showed that SSL-NBV required fewer views for plant reconstruction than non-NBV methods and was over 800 times faster than a voxel-based method. SSL-NBV reduced training annotations by over 90% compared to a baseline DL-NBV. Furthermore, SSL-NBV could adapt to novel scenarios through online fine-tuning. Also using real plants, the results showed that the proposed method can learn to effectively plan new viewpoints for 3D plant reconstruction. Most importantly, SSL-NBV automated the entire network training and uses continuous online learning, allowing it to operate in changing agricultural environments.

3D重建自监督学习机器人农业机器人

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。