用18万张叶病图像和描述训练视觉语言模型，提升农业病害识别准确率
A Vision-Language Foundation Model for Leaf Disease Identification
- 基于18.6万张图像-描述对，用软标签对比学习增强模型泛化能力
- 在零样本、少样本任务中超越CLIP等模型，且参数量不增加
- 适合智能农业、植物病害诊断等需要多模态理解的场景
叶片病害识别在智慧农业中至关重要。然而，现有研究仍难以有效融合图像与文本模态以互补各自缺陷，且多依赖ImageNet等受限数据集预训练，缺乏领域特异性信息。本文提出SCOLD（面向叶病识别的软目标对比学习模型），一个上下文感知的视觉-语言基础模型，专为农业任务设计。SCOLD利用包含超过186,000张植物叶片图像及其对应症状描述的多样化语料库，涵盖97个独特概念。通过任务无关的预训练，该模型采用上下文软标签平滑对比学习中的硬标签，缓解过自信问题，从而提升细粒度分类任务下的泛化性与鲁棒性。实验表明，SCOLD在多个基准测试中优于OpenAI-CLIP-L、BioCLIP和SigLIP2，包括零样本与少样本分类、图文检索及图像分类，同时保持有竞争力的参数规模。消融研究进一步验证了其有效性。该方法显著推动了农业视觉-语言基础模型的发展，可在极少或无需监督微调下实现优异性能。本工作为未来长文本/简化上下文建模、类别模糊任务以及多模态智能植保系统的研究奠定坚实基础。代码已开源：https://huggingface.co/enalis/scold
原文摘要 · Abstract (English)
Leaf disease identification plays a pivotal role in smart agriculture. However, many existing studies still struggle to integrate image and textual modalities to compensate for each other's limitations. Furthermore, many of these approaches rely on pretraining with constrained datasets such as ImageNet, which lack domain-specific information. We propose SCOLD (Soft-target COntrastive learning for Leaf Disease identification), a context-aware vision-language foundation model tailored to address these challenges for agricultural tasks. SCOLD is developed using a diverse corpus of plant leaf images and corresponding symptom descriptions, comprising over 186,000 image-caption pairs aligned with 97 unique concepts. Through task-agnostic pretraining, SCOLD leverages contextual soft targets to mitigate overconfidence in contrastive learning by smoothing labels, thereby improving model generalization and robustness on fine-grained classification tasks. Experimental results demonstrate that SCOLD outperforms existing vision-language models such as OpenAI-CLIP-L, BioCLIP, and SigLIP2 across several benchmarks, including zero-shot and few-shot classification, image-text retrieval, and image classification, while maintaining a competitive parameter footprint. Ablation studies further highlight SCOLD's effectiveness in contrast to its counterparts. The proposed approach significantly advances the agricultural vision-language foundation model, offering strong performance with minimal or no supervised fine-tuning. This work lays a solid groundwork for future research on models trained with long-form and simplified contexts, tasks involving class ambiguity, and multi-modal systems for intelligent plant disease diagnostics. The code for this study is available at https://huggingface.co/enalis/scold
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。