arXiv:2507.01912cs.CV2025-07被引 1

用深学习融合冬夏两季树形数据,解决果园机器视觉遮挡难题。

3D Reconstruction and Information Fusion between Dormant and Canopy Seasons in Commercial Orchards Using Deep Learning and Fast GICP

  • 通过YOLOv9-Seg分割+Kinect Fusion重建,获取高精度树体三维模型。
  • 跨季节对齐误差仅0.00197,重建直径误差低于5.23毫米。
  • 适合果园机器人精准修剪、疏果等自动化作业使用。

在果园自动化中,生长期茂密枝叶严重遮挡树体结构,降低机器视觉对树干、枝条等部件的可见性。而休眠期树木落叶,结构更清晰可见。本文提出一种多季节结构信息融合框架,支持全年作物负载管理。利用YOLOv9-Seg进行实例分割,结合Kinect Fusion实现高分辨率RGB-D图像的3D重建,并采用Fast GICP算法对齐不同季节的模型。通过深度感知掩码提升点云重建精度,验证显示:树干直径RMSE为5.23毫米,枝条直径4.50毫米,枝间距13.72毫米。跨季节配准最小适应度分数达0.00197,实现空间一致的多季节融合建模。该融合结构可揭示生长季被遮挡的关键信息,显著提升修剪、疏果等自动化操作精度。

原文摘要 · Abstract (English)

In orchard automation, dense foliage during the canopy season severely occludes tree structures, minimizing visibility to various canopy parts such as trunks and branches, which limits the ability of a machine vision system. However, canopy structure is more open and visible during the dormant season when trees are defoliated. In this work, we present an information fusion framework that integrates multi-seasonal structural data to support robotic and automated crop load management during the entire growing season. The framework combines high-resolution RGB-D imagery from both dormant and canopy periods using YOLOv9-Seg for instance segmentation, Kinect Fusion for 3D reconstruction, and Fast Generalized Iterative Closest Point (Fast GICP) for model alignment. Segmentation outputs from YOLOv9-Seg were used to extract depth-informed masks, which enabled accurate 3D point cloud reconstruction via Kinect Fusion; these reconstructed models from each season were subsequently aligned using Fast GICP to achieve spatially coherent multi-season fusion. The YOLOv9-Seg model, trained on manually annotated images, achieved a mean squared error (MSE) of 0.0047 and segmentation mAP@50 scores up to 0.78 for trunks in dormant season dataset. Kinect Fusion enabled accurate reconstruction of tree geometry, validated with field measurements resulting in root mean square errors (RMSE) of 5.23 mm for trunk diameter, 4.50 mm for branch diameter, and 13.72 mm for branch spacing. Fast GICP achieved precise cross-seasonal registration with a minimum fitness score of 0.00197, allowing integrated, comprehensive tree structure modeling despite heavy occlusions during the growing season. This fused structural representation enables robotic systems to access otherwise obscured architectural information, improving the precision of pruning, thinning, and other automated orchard operations.

3D重建果园自动化信息融合深度学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。