让视觉语言模型在不同数据分布下都稳定表现，通过对抗训练学出不变提示。
Domain-Invariant Prompt Learning for Vision-Language Models
- 用对抗训练让提示向量适应不同数据分布
- 在多个视觉领域上比CoOp提升性能
- 适合需要跨域泛化的图像分类任务
像CLIP这样的大型预训练视觉语言模型通过将图像和文本对齐到共享特征空间，实现了强大的零样本迁移能力。软提示方法（如上下文优化，CoOp）通过学习一组上下文向量来有效适应下游识别任务。然而，CoOp缺乏处理未见分布间域偏移的显式机制。为此，我们提出域不变上下文优化（DiCoOp），作为CoOp的扩展，专为域泛化设计。通过采用对抗训练策略，DiCoOp迫使模型学习对域变化不敏感的提示，同时保持分类的判别能力。实验结果表明，DiCoOp在多个不同的视觉领域中，域泛化任务上持续优于CoOp。
原文摘要 · Abstract (English)
Large pre-trained vision-language models like CLIP have transformed computer vision by aligning images and text in a shared feature space, enabling robust zero-shot transfer via prompting. Soft-prompting, such as Context Optimization (CoOp), effectively adapts these models for downstream recognition tasks by learning a set of context vectors. However, CoOp lacks explicit mechanisms for handling domain shifts across unseen distributions. To address this, we propose Domain-invariant Context Optimization (DiCoOp), an extension of CoOp optimized for domain generalization. By employing an adversarial training approach, DiCoOp forces the model to learn domain-invariant prompts while preserving discriminative power for classification. Experimental results show that DiCoOp consistently surpasses CoOp in domain generalization tasks across diverse visual domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。