通过视觉语言交互提升害虫细粒度识别,助力农业智能防治
PestVL-Net: Enabling Multimodal Pest Learning via Fine-grained Vision-Language Interaction

- 用RWKV+注意力窗口分割捕捉害虫细微视觉特征
- 结合专家知识与多模态思维链生成精准语义描述
- 适合农业病虫害监测、智能农管系统开发者
有效的害虫识别与管理对可持续农业发展至关重要。然而,真实场景中收集害虫数据往往困难重重。相比其他领域,害虫种类繁多,形态复杂多样。现有技术难以在细粒度层面有效建模害虫的关键视觉特征和高层语义特征,限制了其在实际农业场景中的应用。为此,我们提出一种协同方法,结合PestVL-Net这一新型视觉-语言框架与两个多物种害虫数据集,实现细粒度害虫学习。PestVL-Net的视觉路径采用递归加权键值(RWKV)架构,并引入显著性引导的自适应窗口分割策略,有效捕捉害虫的细粒度视觉特征。同时,语言组件借助多模态大模型(MLLMs)先验,融合农业专家知识,通过多模态思维链(CoT)推理生成精确的害虫语义描述。视觉与文本表征的深度融合实现了细粒度多模态害虫学习。在多个害虫数据集上的实验验证了PestVL-Net的优越性能,凸显其在真实害虫管理中的潜力。
原文摘要 · Abstract (English)
Effective pest recognition and management are crucial for sustainable agricultural development. However, collecting pest data in real scenarios is often challenging. Compared to other domains, pests exhibit a wide variety of species with complex and diverse morphological characteristics. Existing techniques struggle to effectively model the key visual and high-level semantic features of pests in a fine-grained manner. These limitations hinder the practical application of such methods in real agricultural scenarios. To address these critical challenges, we present a synergistic approach that integrates PestVL-Net, a novel vision-language framework, with two multi-species pest datasets to facilitate fine-grained pest learning. The visual pathway of PestVL-Net utilizes the Recurrent Weighted Key Value (RWKV) architecture, incorporating a saliency-guided adaptive window partitioning scheme to effectively model the fine-grained visual characteristics of pests. Concurrently, the linguistic component generates precise pest semantic descriptions by leveraging Multimodal Large Language Models (MLLMs) priors, critically informed by agricultural expert knowledge and structured via multimodal Chain-of-Thought (CoT) reasoning. The deep fusion of these complementary visual and textual representations enables fine-grained multimodal pest learning. Extensive experimental evaluations on multiple pest datasets validate the superior performance of PestVL-Net, highlighting its potential for effective real-world pest management.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。