arXiv:2411.07799cs.CVcs.RO2024-11被引 2

用彩色点云实现果树果实跨时精准追踪,解决大小形状变化难题

Horticultural Temporal Fruit Monitoring via 3D Instance Segmentation and Re-Identification using Colored Point Clouds

  • 直接在彩色点云上做实例分割与特征提取,保留精细3D结构
  • 通过注意力匹配网络实现跨时间果实关联,准确率显著提升
  • 适用于草莓苹果等复杂果园,对遮挡和动态变化鲁棒

精确且一致的果实长期监测是实现自动化农业系统的关键。但受果实大小、形状、遮挡、朝向差异及果园动态变化(如果实增减)影响,该任务极具挑战。本文提出一种基于地基彩色点云的果实实例分割与跨时重识别方法。直接在密集彩色点云上进行学习型实例分割,捕捉细粒度三维空间信息。对每个分割出的果实,利用3D稀疏卷积神经网络提取紧凑且具有区分性的特征描述符。为实现跨时段追踪,设计基于注意力的匹配网络,采用概率分配策略选择最可能的跨时对应关系。在真实草莓和苹果数据集上验证,该方法在实例分割与时间重识别任务中均优于现有技术,可在复杂动态果园环境中实现鲁棒、精确的果实监测。

原文摘要 · Abstract (English)

Accurate and consistent fruit monitoring over time is a key step toward automated agricultural production systems. However, this task is inherently difficult due to variations in fruit size, shape, occlusion, orientation, and the dynamic nature of orchards where fruits may appear or disappear between observations. In this article, we propose a novel method for fruit instance segmentation and re-identification on 3D terrestrial point clouds collected over time. Our approach directly operates on dense colored point clouds, capturing fine-grained 3D spatial detail. We segment individual fruits using a learning-based instance segmentation method applied directly to the point cloud. For each segmented fruit, we extract a compact and discriminative descriptor using a 3D sparse convolutional neural network. To track fruits across different times, we introduce an attention-based matching network that associates fruits with their counterparts from previous sessions. Matching is performed using a probabilistic assignment scheme, selecting the most likely associations across time. We evaluate our approach on real-world datasets of strawberries and apples, demonstrating that it outperforms existing methods in both instance segmentation and temporal re-identification, enabling robust and precise fruit monitoring across complex and dynamic orchard environments. Keywords = Agricultural Robotics, 3D Fruit Tracking, Instance Segmentation, Deep Learning , Point Clouds, Sparse Convolutional Networks, Temporal Monitoring

3D追踪点云分析农业机器人实例分割

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。