arXiv:2506.16994cs.CVcs.LG2025-06

用提示词实现零样本域适应,轻量高效适配无人机等低资源场景。

Prmpt2Adpt: Prompt-Based Zero-Shot Domain Adaptation for Resource-Constrained Environments

  • 通过提示词引导特征对齐,仅用少量源数据实现域自适应。
  • 在MDS-A上达到顶尖检测性能,推理速度提升5倍,适应快7倍。
  • 适合部署在内存计算受限的实时视觉系统,如无人机、移动设备。

无监督域适应(UDA)是真实视觉系统中的关键挑战,尤其在无人机等资源受限环境中,内存与计算能力有限。现有基于提示的UDA方法通常依赖大型视觉语言模型,且需完整访问源域数据,限制了实际应用。本文提出Prmpt2Adpt,一种轻量高效的零样本域适应框架,采用教师-学生范式,由提示驱动的特征对齐引导。核心为微调并蒸馏的CLIP模型,作为Faster R-CNN教师模型的冻结主干。仅使用少量低层源特征,通过提示驱动实例归一化(PIN)将其对齐至目标域语义(仅通过自然语言提示指定)。这些语义引导特征用于短时微调教师模型的检测头。适配后的教师生成高质量伪标签,指导紧凑学生模型的在线适应。在MDS-A数据集上的实验表明,Prmpt2Adpt性能媲美前沿方法,同时实现最高7倍的适应加速和5倍的推理提速,仅需少量源图像,为低资源环境下的实时自适应提供了实用且可扩展的解决方案。

原文摘要 · Abstract (English)

Unsupervised Domain Adaptation (UDA) is a critical challenge in real-world vision systems, especially in resource-constrained environments like drones, where memory and computation are limited. Existing prompt-driven UDA methods typically rely on large vision-language models and require full access to source-domain data during adaptation, limiting their applicability. In this work, we propose Prmpt2Adpt, a lightweight and efficient zero-shot domain adaptation framework built around a teacher-student paradigm guided by prompt-based feature alignment. At the core of our method is a distilled and fine-tuned CLIP model, used as the frozen backbone of a Faster R-CNN teacher. A small set of low-level source features is aligned to the target domain semantics-specified only through a natural language prompt-via Prompt-driven Instance Normalization (PIN). These semantically steered features are used to briefly fine-tune the detection head of the teacher model. The adapted teacher then generates high-quality pseudo-labels, which guide the on-the-fly adaptation of a compact student model. Experiments on the MDS-A dataset demonstrate that Prmpt2Adpt achieves competitive detection performance compared to state-of-the-art methods, while delivering up to 7x faster adaptation and 5x faster inference speed using few source images-making it a practical and scalable solution for real-time adaptation in low-resource domains.

域适应轻量化提示工程实时检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。