用合成图像替代文本描述,实现高效零样本域适应
SIDA: Synthetic Image Driven Zero-shot Domain Adaptation
- 用合成图像模拟目标域风格,避免依赖文本描述
- 在多个挑战性域上达到领先性能,适应时间大幅缩短
- 适合需要快速部署、无目标图像数据的场景
零样本域适应旨在不使用目标域图像的情况下使模型适应新域。现有方法依赖CLIP嵌入空间和文本描述来模拟目标域风格特征,但这类文本驱动方法难以捕捉复杂真实变化,且因对齐过程导致适应时间显著增加。本文提出SIDA,一种新颖高效的零样本域适应方法,利用合成图像实现适应。首先生成与源域相似的图像,并通过图像翻译融入目标域风格;随后以这些合成图像的风格特征作为目标域代理。基于此,引入域混合(Domain Mix)和局部风格迁移(Patch Style Transfer)模块,分别通过融合多种风格扩展域内表示,以及为图像不同区域分配不同风格,有效建模真实世界变化。实验表明,SIDA在多样化的零样本适应场景中表现优异,尤其在困难域上超越现有方法,同时大幅降低整体适应时间。
原文摘要 · Abstract (English)
Zero-shot domain adaptation is a method for adapting a model to a target domain without utilizing target domain image data. To enable adaptation without target images, existing studies utilize CLIP's embedding space and text description to simulate target-like style features. Despite the previous achievements in zero-shot domain adaptation, we observe that these text-driven methods struggle to capture complex real-world variations and significantly increase adaptation time due to their alignment process. Instead of relying on text descriptions, we explore solutions leveraging image data, which provides diverse and more fine-grained style cues. In this work, we propose SIDA, a novel and efficient zero-shot domain adaptation method leveraging synthetic images. To generate synthetic images, we first create detailed, source-like images and apply image translation to reflect the style of the target domain. We then utilize the style features of these synthetic images as a proxy for the target domain. Based on these features, we introduce Domain Mix and Patch Style Transfer modules, which enable effective modeling of real-world variations. In particular, Domain Mix blends multiple styles to expand the intra-domain representations, and Patch Style Transfer assigns different styles to individual patches. We demonstrate the effectiveness of our method by showing state-of-the-art performance in diverse zero-shot adaptation scenarios, particularly in challenging domains. Moreover, our approach achieves high efficiency by significantly reducing the overall adaptation time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。