用仿真生成数据实现果树分段的零样本联合分割,效率更高。
Joint 3D Point Cloud Segmentation using Real-Sim Loop: From Panels to Trees and Branches

- 通过仿真生成真实果园点云数据,构建层次化训练集
- 模型参数减少40%仍优于主流方法,零样本迁移效果好
- 适合果园机器人与数字孪生系统开发人员使用
现代果园采用规则行距和板区分区以提升管理效率。准确高效的点云联合分割(从板到树再到枝)对机器人作业至关重要。然而现有方法多为单实例分割,依赖多个深度网络串联完成任务,难以利用数据中的层次信息,导致误差累积和标注、计算成本上升,限制了实际应用的可扩展性。本文提出一种新方法,结合Real2Sim L-TreeGen生成训练数据与专为P2TB任务设计的联合模型J-P2TB。该模型在仿真数据上训练后,通过零样本学习直接应用于真实果园点云的联合分割。相比代表性方法,本模型在多数分割指标上表现更优,且参数量减少40%。该仿真到现实的结果验证了L-TreeGen在训练中的有效性及J-P2TB在联合分割中的高精度、高效性与强泛化能力,不仅推动果园机器人自动化发展,也助力数字孪生技术进步。
原文摘要 · Abstract (English)
Modern orchards are planted in structured rows with distinct panel divisions to improve management. Accurate and efficient joint segmentation of point cloud from Panel to Tree and Branch (P2TB) is essential for robotic operations. However, most current segmentation methods focus on single instance segmentation and depend on a sequence of deep networks to perform joint tasks. This strategy hinders the use of hierarchical information embedded in the data, leading to both error accumulation and increased costs for annotation and computation, which limits its scalability for real-world applications. In this study, we proposed a novel approach that incorporated a Real2Sim L-TreeGen for training data generation and a joint model (J-P2TB) designed for the P2TB task. The J-P2TB model, trained on the generated simulation dataset, was used for joint segmentation of real-world panel point clouds via zero-shot learning. Compared to representative methods, our model outperformed them in most segmentation metrics while using 40% fewer learnable parameters. This Sim2Real result highlighted the efficacy of L-TreeGen in model training and the performance of J-P2TB for joint segmentation, demonstrating its strong accuracy, efficiency, and generalizability for real-world applications. These improvements would not only greatly benefit the development of robots for automated orchard operations but also advance digital twin technology.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。