用跨模态流匹配弥合图像与文本嵌入的差距，提升模型在风格变化下的泛化能力。
CrossFlowDG: Bridging the Modality Gap with Cross-modal Flow Matching for Domain Generalization
- 在联合欧氏空间中学习图像到文本的连续变换，直接对齐跨模态特征
- 在TerraIncognita上达到当前最佳性能，其他基准也表现竞争力
- 适合关注多模态融合与领域泛化的研究者使用
领域泛化（DG）旨在应对域偏移问题，其在计算机视觉中主要表现为风格差异导致模型过度依赖特定域的外观特征而非类别语义。现有方法虽利用文本表示作为稳定的域不变锚点，但基于余弦相似度的对比对齐仍存在模态间隙，即图像与文本嵌入在几何上保持分离。本文提出CrossFlowDG，一种新颖的领域泛化框架，通过无噪声的跨模态流匹配解决这一残余间隙。该框架在联合欧氏潜空间中学习连续变换，将带有域偏差的图像嵌入显式地传输至对应类别的域不变文本嵌入。采用高效的VMamba图像编码器和CLIP文本编码器，在四个常见DG基准上进行测试，表现优异，尤其在TerraIncognita上达到当前最优。代码已开源。
原文摘要 · Abstract (English)
Domain generalization (DG) aims to maintain performance under domain shift, which in computer vision appears primarily as stylistic variations that cause models to overfit to domain-specific appearance cues rather than class semantics. To overcome this, recent methods use textual representations as stable, domain-invariant anchors. However, multimodal approaches that rely on cosine similarity-based contrastive alignment leave a modality gap where image and text embeddings remain geometrically separated despite semantic correspondence. We propose CrossFlowDG, a novel DG framework that addresses this residual gap using noise-free, cross-modal flow matching. By learning a continuous transformation in the joint Euclidean latent space, our framework explicitly transports domain-biased image embeddings toward domain-invariant text embeddings of the correct class. Using the efficient VMamba image encoder and CLIP's text encoder, CrossFlowDG is tested against four common DG benchmarks, and achieves competitive performance on several benchmarks and state-of-the-art on TerraIncognita. Code is available at: https://github.com/ajkrit/CrossFlowDG
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。