用神经辐射场与对比学习实现多类水果无差别计数
FruitNeRF++: A Generalized Multi-Fruit Counting Method Utilizing Contrastive Learning and Neural Radiance Fields
- 融合视觉基础模型掩码，构建无形状依赖的果实实例编码
- 在合成与真实苹果数据集上达到领先计数精度
- 适合果园自动化监测与多物种果实统计场景
我们提出FruitNeRF++，一种结合对比学习与神经辐射场的新颖果实计数方法，可从果园非结构化照片中计数多种果实。该方法基于FruitNeRF，采用神经语义场与果实特异性聚类策略，但需针对每种果实类型单独适配，限制了实用性。为此，我们设计了一种形状无关的多果实计数框架，将RGB与语义数据与视觉基础模型预测的实例掩码结合，利用掩码将每个果实的身份编码为实例嵌入，并存入神经实例场。通过体积采样神经场，提取嵌入实例特征的点云，可进行无果实种类依赖的聚类以获得计数结果。我们在包含苹果、李子、柠檬、梨、桃和芒果的合成数据集以及真实苹果基准数据集上评估本方法，结果表明FruitNeRF++更易控制，且优于其他先进方法。
原文摘要 · Abstract (English)
We introduce FruitNeRF++, a novel fruit-counting approach that combines contrastive learning with neural radiance fields to count fruits from unstructured input photographs of orchards. Our work is based on FruitNeRF, which employs a neural semantic field combined with a fruit-specific clustering approach. The requirement for adaptation for each fruit type limits the applicability of the method, and makes it difficult to use in practice. To lift this limitation, we design a shape-agnostic multi-fruit counting framework, that complements the RGB and semantic data with instance masks predicted by a vision foundation model. The masks are used to encode the identity of each fruit as instance embeddings into a neural instance field. By volumetrically sampling the neural fields, we extract a point cloud embedded with the instance features, which can be clustered in a fruit-agnostic manner to obtain the fruit count. We evaluate our approach using a synthetic dataset containing apples, plums, lemons, pears, peaches, and mangoes, as well as a real-world benchmark apple dataset. Our results demonstrate that FruitNeRF++ is easier to control and compares favorably to other state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。