用CLIP提示词解决多目标开放集域适应问题
COSMo: CLIP Talks on Open-Set Multi-Target Domain Adaptation
- 在提示词空间学习与领域无关的提示，通过源域引导
- 在三个数据集上平均提升5.1%，优于现有方法
- 首次解决开放集多目标域适应，适合真实场景应用
多目标域适应（MTDA）旨在从单一源域学习领域不变信息，并应用于多个未标注目标域。现有方法多关注视觉特征域偏移，忽视语义特征，难以处理未知类别，导致开放集多目标域适应（OSMTDA）挑战。尽管像CLIP这样的大规模视觉-语言基础模型展现潜力，其在MTDA中的应用仍待探索。本文提出COSMo，通过源域引导的提示学习，在提示空间中构建领域无关提示，结合领域特异性偏差网络和已知/未知类别的独立提示，有效应对领域与类别偏移。据我们所知，COSMo是首个解决开放集多目标域适应的方法，更贴近真实场景。在Mini-DomainNet、Office-31和Office-Home三个挑战性数据集上，相比适配至OSMTDA设置的其他方法，平均提升5.1%。代码已公开。
原文摘要 · Abstract (English)
Multi-Target Domain Adaptation (MTDA) entails learning domain-invariant information from a single source domain and applying it to multiple unlabeled target domains. Yet, existing MTDA methods predominantly focus on addressing domain shifts within visual features, often overlooking semantic features and struggling to handle unknown classes, resulting in what is known as Open-Set (OS) MTDA. While large-scale vision-language foundation models like CLIP show promise, their potential for MTDA remains largely unexplored. This paper introduces COSMo, a novel method that learns domain-agnostic prompts through source domain-guided prompt learning to tackle the MTDA problem in the prompt space. By leveraging a domain-specific bias network and separate prompts for known and unknown classes, COSMo effectively adapts across domain and class shifts. To the best of our knowledge, COSMo is the first method to address Open-Set Multi-Target DA (OSMTDA), offering a more realistic representation of real-world scenarios and addressing the challenges of both open-set and multi-target DA. COSMo demonstrates an average improvement of $5.1\%$ across three challenging datasets: Mini-DomainNet, Office-31, and Office-Home, compared to other related DA methods adapted to operate within the OSMTDA setting. Code is available at: https://github.com/munish30monga/COSMo
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。