分阶段逐步对齐,提升多源无监督图像迁移效果
Multi-Prompt Progressive Alignment for Multi-Source Unsupervised Domain Adaptation
- 先用高置信度样本训练,再逐步加入难样本
- 在ImageCLEF等三个基准上达顶尖性能
- 适合处理噪声大、来源多的现实迁移任务
大型视觉语言模型如CLIP凭借强大的零样本泛化能力,已成为无监督域适应(UDA)的重要基础。现有方法通常利用CLIP为目标域生成伪标签后统一微调模型以学习域不变特征,但一次性对齐所有伪标签数据会受噪声和难分类样本影响,导致错误传播与学习效果下降。该问题在多源场景下更严重,因多个源域间差异大、噪声水平不一,进一步加剧对齐不稳定性。为此,本文提出一种渐进式对齐策略MP^2A:先在目标域高置信度样本上训练,建立可靠表示;随后逐步引入更具挑战性的样本,引导模型在不被初始噪声干扰的前提下持续优化。该方法有效缓解确认偏见,促进更稳健的收敛,实现真正域不变特征的学习。在ImageCLEF、Office-Home及最具挑战性的DomainNet三个主流UDA基准上验证,MP^2A优于近期多数基于CLIP的多源无监督域适应方法,充分证明其有效性。
原文摘要 · Abstract (English)
Large Vision-Language Models like CLIP have become a powerful foundation for Unsupervised Domain Adaptation due to their strong zero-shot generalization. State-of-the-art methods typically leverage CLIP to generate pseudo-labels for the target domain, then fine-tune the model to learn domain-invariant features. However, these methods attempt to align source and target domains using all pseudo-labeled data simultaneously. This one-shot alignment struggles with noisy, hard-to-classify samples, leading to error propagation and suboptimal feature learning. The problem is even more amplified in the multi-source scenario, where diverse domain gaps and varying noise levels across multiple source domains further destabilize the alignment process. To address this issue, in this work, we propose a progressive alignment strategy for adapting CLIP to unlabeled downstream task. Our method begins by training the model on a high-confidence subset of target samples, allowing it to first learn a well-aligned representation from the most reliable data. As training progresses, it gradually incorporates more challenging samples, guiding the model to refine its understanding without being overwhelmed by initial label noise. This progressive approach effectively mitigates confirmation bias and promotes a more robust convergence, allowing for the learning of genuinely domain-invariant features. We name our approach MP^2A and test it on three popular UDA benchmarks, namely ImageCLEF, Office-Home, and the most challenging DomainNet. Experiments showcase that MP^2A achieves state-of-the-art performance when compared with most recent CLIP-based MS-UDA approaches, demonstrating the effectiveness of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。