用可迁移性感知的Transformer提升无监督域适应性能
Feature Fusion Transferability Aware Transformer for Unsupervised Domain Adaptation
- 引入补丁判别器生成可迁移性矩阵，引导模型关注可迁移特征
- 在隐空间融合特征，让每个嵌入获取全局信息，增强泛化能力
- 在多个基准上达最优，适合做视觉领域自适应研究
无监督域适应（UDA）旨在利用带标签源域的知识提升无标签目标域的性能。尽管卷积神经网络（CNN）在以往的UDA方法中占主导地位，但近期研究显示视觉变换器（ViTs）在此任务中也展现出潜力。本文提出一种新型特征融合可迁移性感知变换器（FFTAT），以增强ViT在UDA任务中的表现。方法包含两项关键创新：首先，引入补丁判别器评估补丁的可迁移性，生成可迁移性矩阵，并将其融入自注意力机制，引导模型聚焦于可迁移补丁；其次，提出一种潜在空间特征融合技术，使每个嵌入能融合其他所有嵌入的信息，从而提升泛化能力。两者协同作用，显著增强特征表示学习。在多个主流基准上的大量实验表明，该方法显著提升UDA性能，达到当前最优（SOTA）结果。
原文摘要 · Abstract (English)
Unsupervised domain adaptation (UDA) aims to leverage the knowledge learned from labeled source domains to improve performance on the unlabeled target domains. While Convolutional Neural Networks (CNNs) have been dominant in previous UDA methods, recent research has shown promise in applying Vision Transformers (ViTs) to this task. In this study, we propose a novel Feature Fusion Transferability Aware Transformer (FFTAT) to enhance ViT performance in UDA tasks. Our method introduces two key innovations: First, we introduce a patch discriminator to evaluate the transferability of patches, generating a transferability matrix. We integrate this matrix into self-attention, directing the model to focus on transferable patches. Second, we propose a feature fusion technique to fuse embeddings in the latent space, enabling each embedding to incorporate information from all others, thereby improving generalization. These two components work in synergy to enhance feature representation learning. Extensive experiments on widely used benchmarks demonstrate that our method significantly improves UDA performance, achieving state-of-the-art (SOTA) results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。