对比零样本SAM3与微调YOLO在密集苹果分割中的表现
Generalization vs. Specialization: Evaluating Segment Anything Model (SAM3) Zero-Shot Segmentation Against Fine-Tuned YOLO Detectors
- 用零样本SAM3和微调YOLO11对比分割性能
- 在0.15 IoU下YOLO F1达68.9%-72.2%，SAM3为59.8%
- SAM3边界稳定性强，性能下降仅4点，是YOLO的12倍
深度学习推动了实例分割的两种范式：通过任务微调优化的专用模型与具备零样本分割能力的通用基础模型。本文系统比较了零样本模式下的SAM3（Segment Anything Model，又称SAMv3）与三种微调后的Ultralytics YOLO11（nano、medium、large）模型。评估基于包含670张果园图像和28,179个标注苹果实例的MinneApple数据集，该数据集具有高密度与遮挡特性，可严格验证模型在复杂场景下的表现。分析显示，IoU选择可使性能差距夸大高达30%。在合适的IoU=0.15阈值下，YOLO模型分别取得68.9%、72.2%和71.9%的F1分数，而纯零样本的SAM3为59.8%。然而，当IoU变化时，YOLO性能下降48-50分，而SAM3仅下降4分，表明其边界稳定性高出12倍。这凸显了SAMv3在掩码精度上的优势，以及YOLO11在检测完整性的专长。我们开源了代码、评估流程与方法建议，助力理解专用模型与通用模型在密集实例分割任务中的适用场景。
原文摘要 · Abstract (English)
Deep learning has advanced two fundamentally different paradigms for instance segmentation: specialized models optimized through task-specific fine-tuning and generalist foundation models capable of zero-shot segmentation. This work presents a comprehensive comparison between SAM3 (Segment Anything Model, also called SAMv3) operating in zero-shot mode and three variants of Ultralytics YOLO11 (nano, medium, and large) fine-tuned for instance segmentation. The evaluation is conducted on the MinneApple dataset, a dense benchmark comprising 670 orchard images with 28,179 annotated apple instances, enabling rigorous validation of model behavior under high object density and occlusion. Our analysis shows IoU choices can inflate performance gaps by up to 30%. At the appropriate IoU = 0.15 threshold, YOLO models achieve 68.9%, 72.2%, and 71.9% F1, while SAM3 reaches 59.8% in pure zero-shot mode. However, YOLO exhibits steep degradation 48-50 points across IoU ranges whereas SAM3 drops only 4 points, revealing 12 times superior boundary stability of SAM3. This highlights the strength of SAMv3 in mask precision versus specialization in detection completeness of YOLO11. We provide open-source code, evaluation pipelines, and methodological recommendations, contributing to a deeper understanding of when specialized fine-tuned models or generalist foundation models are preferable for dense instance segmentation tasks. This project repository is available on GitHub as https://github.com/Applied-AI-Research-Lab/Segment-Anything-Model-SAM3-Zero-Shot-Segmentation-Against-Fine-Tuned-YOLO-Detectors
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。