用大模型精准识别苹果成熟度和大小,实现精准采摘。
Foundation Model-Based Apple Ripeness and Size Estimation for Selective Harvesting
- 基于视觉大模型检测苹果并判断成熟度
- 构建4027张图像的苹果数据集,准确率超现有模型
- 适合果园自动化采摘系统研发者参考
采摘是果树产业中的关键环节,依赖大量人工劳动且成本高,还存在安全隐患。近年来自动化采摘技术为高效、低成本、安全的果实采收提供了可能,但现有技术常不分成熟度与大小,对所有可见果实一概采收。本文提出一种基于基础模型的苹果成熟度与大小估计框架,构建了包含4,027张图像和16,257个标注苹果的公开数据集Fuji-Ripeness-Size Dataset,新增基于果色和拍摄日期的成熟度标签(“成熟”与“未成熟”)。利用基于语言模型的检测器Grounding-DINO,实现鲁棒的苹果检测与成熟度分类,性能优于其他先进模型。同时评估六种大小估计算法,选出误差与方差最小的最优方案。该数据集及算法已开源,为未来自动与选择性采摘研究提供重要基准。
原文摘要 · Abstract (English)
Harvesting is a critical task in the tree fruit industry, demanding extensive manual labor and substantial costs, and exposing workers to potential hazards. Recent advances in automated harvesting offer a promising solution by enabling efficient, cost-effective, and ergonomic fruit picking within tight harvesting windows. However, existing harvesting technologies often indiscriminately harvest all visible and accessible fruits, including those that are unripe or undersized. This study introduces a novel foundation model-based framework for efficient apple ripeness and size estimation. Specifically, we curated two public RGBD-based Fuji apple image datasets, integrating expanded annotations for ripeness ("Ripe" vs. "Unripe") based on fruit color and image capture dates. The resulting comprehensive dataset, Fuji-Ripeness-Size Dataset, includes 4,027 images and 16,257 annotated apples with ripeness and size labels. Using Grounding-DINO, a language-model-based object detector, we achieved robust apple detection and ripeness classification, outperforming other state-of-the-art models. Additionally, we developed and evaluated six size estimation algorithms, selecting the one with the lowest error and variation for optimal performance. The Fuji-Ripeness-Size Dataset and the apple detection and size estimation algorithms are made publicly available, which provides valuable benchmarks for future studies in automated and selective harvesting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。