用无人机图像检测刺菜,ViT模型比传统CNN更抗域偏移。
Towards Robust Deep Learning-based Rumex Obtusifolius Detection from Drone Images

- 用自监督预训练的ViT天然适应域变化,无需额外适配
- 在瑞士牧场15架次无人机数据上,F1达0.8
- 公开了含15次飞行数据的AGSMultiRumex数据集
领域自适应(DA)旨在将源域上训练的模型迁移至数据分布不同的目标域。本文研究刺菜(Rumex obtusifolius)图像分类中的域适应问题:在基于地面车辆采集的公开数据集(源域)上训练模型,评估其在无人机(UAV)采集的自建目标数据集上的表现。结果发现,即使经过微调,卷积神经网络(如ResNet)在目标域上泛化能力仍差。采用均值匹配和最大分类器差异两种经典域适应方法可显著提升性能。而经过自监督预训练的视觉变换器(ViT,如DINOv2、DINOv3)能自然应对域偏移,性能超过经域适应优化的ResNet,归因于大规模预训练获得的通用表征。在源域微调后,使用ViT在目标域上实现高达F1=0.8的分类性能。为支持草地杂草检测的域适应研究,本文公开了基于无人机的靶向数据集AGSMultiRumex,包含15次瑞士牧场飞行采集的数据。
原文摘要 · Abstract (English)
Domain adaptation (DA) addresses the challenge of transferring a machine learning model trained on a source domain to a target domain with a different data distribution. In this work, we study DA for the task of Rumex obtusifolius (Rumex) image classification. We train models on a published, ground vehicle-based dataset (source) and evaluate their performance on a custom target dataset acquired by unmanned aerial vehicles (UAVs). We find that Convolutional Neural Network (CNN) models, specifically ResNets, generalize poorly to the target domain, even after fine-tuning on the source data. Applying moment-matching and maximum classifier discrepancy, two established DA techniques, substantially improves target-domain performance. However, Vision Transformer (ViT) models pretrained with self-supervised objectives (DINOv2, DINOv3) handle domain shifts intrinsically well, surpassing even moment-matching-trained ResNets, likely due to the rich, general-purpose representations acquired during large-scale pretraining. Using ViTs fine-tuned on the source dataset, we demonstrate high classification performances in the range of F1=0.8 on our target dataset. To support further research on DA for weed detection in grassland systems, we publicly release our UAV-based target dataset AGSMultiRumex, comprising data from 15 flights over Swiss meadows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。