arXiv:2505.24026cs.CVcs.AI2025-05CVPR被引 4

用深度信息提升农田杂草分割,无需标注就能跨田地准确识别

MaskAdapt: Unsupervised Geometry-Aware Domain Adaptation Using Multimodal Contextual Learning and RGB-Depth Masking

  • 融合RGB与深度图梯度,通过交叉注意力优化特征表示
  • 在真实农田数据上实现比现有最优方法更高的分割mIOU
  • 适合农业自动化、无人农场等需要无监督泛化的场景

作物与杂草的语义分割对精准农事管理至关重要,但现有方法依赖耗时的像素级标注。当模型在某地块(源域)训练后应用于新地块(目标域)时,因光照、相机设置、土壤成分和作物生长阶段差异导致的域偏移会降低性能。无监督域适应(UDA)可解决此问题,但当前方法在遮挡和作物与杂草视觉混杂时易出错。为此,我们提出MaskAdapt,通过整合RGB图像与深度数据特征,利用深度图梯度捕捉空间变化以缓解纹理模糊。梯度经交叉注意力机制用于细化RGB特征,增强边界清晰度。同时,设计几何感知掩码策略,在训练中施加水平、垂直及随机掩码,促使模型关注整体空间上下文。在真实农业数据集上的评估表明,MaskAdapt持续优于现有SOTA UDA方法,显著提升不同田间条件下的分割平均交并比(mIOU)。

原文摘要 · Abstract (English)

Semantic segmentation of crops and weeds is crucial for site-specific farm management; however, most existing methods depend on labor intensive pixel-level annotations. A further challenge arises when models trained on one field (source domain) fail to generalize to new fields (target domain) due to domain shifts, such as variations in lighting, camera setups, soil composition, and crop growth stages. Unsupervised Domain Adaptation (UDA) addresses this by enabling adaptation without target-domain labels, but current UDA methods struggle with occlusions and visual blending between crops and weeds, leading to misclassifications in real-world conditions. To overcome these limitations, we introduce MaskAdapt, a novel approach that enhances segmentation accuracy through multimodal contextual learning by integrating RGB images with features derived from depth data. By computing depth gradients from depth maps, our method captures spatial transitions that help resolve texture ambiguities. These gradients, through a cross-attention mechanism, refines RGB feature representations, resulting in sharper boundary delineation. In addition, we propose a geometry-aware masking strategy that applies horizontal, vertical, and stochastic masks during training. This encourages the model to focus on the broader spatial context for robust visual recognition. Evaluations on real agricultural datasets demonstrate that MaskAdapt consistently outperforms existing State-of-the-Art (SOTA) UDA methods, achieving improved segmentation mean Intersection over Union (mIOU) across diverse field conditions.

语义分割无监督学习农业AI多模态融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。