arXiv:2510.01660cs.CV2025-10被引 1

用视觉重编程层让预训练模型跨域复用,参数少、效果好。

VirDA: Reusing Backbone for Unsupervised Domain Adaptation with Visual Reprogramming

  • 在主干网络前加领域专属重编程层,生成风格化提示图
  • 仅需150万参数即达92.8%准确率,比主流方法少98%参数
  • 无需修改主干参数,可通用多个领域,适合资源受限场景

现有无监督域自适应方法对每组源-目标域都微调完整主干网络,导致参数与存储随新域线性增长,且无法复用已训练主干。受现有主干存在纹理偏见的启发,我们提出通过视觉重编程(VirDA)利用领域特定纹理偏见进行域自适应。与微调整个主干不同,VirDA 在主干前添加一个领域专属的视觉重编程层,该层生成视觉提示,作为输入图像的额外纹理偏见,使其风格适配目标域。通过多目标函数优化这些重编程层,以最小化应用后视觉提示的域内与域间分布差异。该过程无需修改主干参数,使同一主干可在不同域间复用。在 Office-31 上评估,仅用 150 万可训练参数即达 92.8% 平均准确率。相比当前最优的参数高效基线 PDA 提升 1.6%,仅使用其 46% 参数;相比全主干微调的 CDTrans 与 FixBi 分别提升 0.2% 和 1.4%,却仅需其 1.7% 和 2.8% 的可训练参数。相较最强方法 PMTrans 与 TVT,VirDA 仅用约 1.7% 参数,准确率损失分别为 2.2% 和 1.1%。

原文摘要 · Abstract (English)

Existing UDA pipelines fine-tune already well-trained backbone parameters for every new source-and-target pair, resulting in the number of training parameters and storage memory growing linearly with each new pair, and also preventing the reuse of these well-trained backbone parameters. Inspired by recent implications that existing backbones have textural biases, we propose making use of domain-specific textural bias for domain adaptation via visual reprogramming, namely VirDA. Instead of fine-tuning the full backbone, VirDA prepends a domain-specific visual reprogramming layer to the backbone. This layer produces visual prompts that act as an added textural bias to the input image, adapting its "style" to a target domain. To optimize these visual reprogramming layers, we use multiple objective functions that optimize the intra- and inter-domain distribution differences when domain-adapting visual prompts are applied. This process does not require modifying the backbone parameters, allowing the same backbone to be reused across different domains. We evaluate VirDA on Office-31 and obtain 92.8% mean accuracy with only 1.5M trainable parameters. VirDA surpasses PDA, the state-of-the-art parameter-efficient UDA baseline, by +1.6% accuracy while using just 46% of its parameters. Compared with full-backbone fine-tuning, VirDA outperforms CDTrans and FixBi by +0.2% and +1.4%, respectively, while requiring only 1.7% and 2.8% of their trainable parameters. Relative to the strongest current methods (PMTrans and TVT), VirDA uses ~1.7% of their parameters and trades off only 2.2% and 1.1% accuracy, respectively.

域自适应视觉重编程参数效率模型复用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。