针对无人机图像检测,提出高效微调方法,显著提升视觉语言模型性能。
DroneFINE: Domain-Aware Parameter-Efficient Fine-Tuning of Vision-Language Detectors for Drone Images

- 设计动态多路径适配器与语义抑制模块,贴合无人机图像特点。
- 在VisDrone和UAVDT上超越现有参数高效微调方法,接近全量微调效果。
- 适合需要轻量化部署的无人机视觉检测场景,兼顾精度与效率。
在开放动态环境中运行的无人机(UAV)目标检测任务极具挑战性。尽管视觉语言模型(VLM)为通用目标检测提供了强大解决方案,但其预训练数据与航拍图像之间存在显著领域差异,导致现有参数高效微调(PEFT)方法难以有效适配。原因在于VLM的“自然场景、前景主导”视觉先验与无人机图像“鸟瞰视角、背景主导、小目标”特征严重不匹配。为此,本文提出DroneFINE,一种面向无人机场景的新型参数高效微调范式,包含两个领域感知的互补模块:一是数据依赖、前景感知、多路径的自适应机制HyperAdapter,突破传统PEFT静态结构限制;二是基于文本引导的背景抑制算法SemanticGate,利用背景词汇主动抑制无关区域响应。在VisDrone和UAVDT数据集上的大量实验表明,DroneFINE显著优于现有PEFT方法,在仅使用极少可训练参数的情况下,性能接近全量微调。
原文摘要 · Abstract (English)
Object detection for Unmanned Aerial Vehicles (UAVs) working in open and dynamic environments is a highly challenging task. While Vision-Language Models (VLMs) have offered a powerful solution for universal object detection, adapting them to UAV scenarios remains non-trivial due to a substantial domain gap between VLM pre-training data and aerial imagery. The prevailing Parameter-Efficient Fine-Tuning (PEFT) methods prove ineffective in bridging this gap, as VLMs' "natural-scene, foreground-dominant" visual priors misalign with the "bird's-eye-view, background-dominant, small-object" characteristics of UAV data. To address this issue, we propose DroneFINE, a novel PEFT paradigm comprising two domain-aware complementary modules tailored for VLM-based drone image detectors. Specifically, a data-dependent, foreground-aware, and multi-path adaptation mechanism named HyperAdapter is designed, which overcomes the static structural constraints of PEFT. In addition, a background suppression algorithm named SemanticGate is developed. It is a text-conditioned guidance strategy that employs background vocabulary to actively guide the model in suppressing responses from irrelevant regions. Extensive experiments on VisDrone and UAVDT demonstrate that DroneFINE significantly outperforms existing PEFT methods and achieves performance comparable to full fine-tuning while substantially reducing the number of trainable parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。