通过消除风格差异提升跨域泛化能力，让模型更适应新场景。
SHED: Style-Homogenized Embedding Alignment for Domain Generalization
- 用风格归一化的嵌入替代原始特征进行对齐
- 在五个基准上达到当前最优，提升最高达4.0%
- 适合需要跨域鲁棒性的实际应用
领域泛化旨在提升模型对未见领域的鲁棒性，应对嵌入分布偏移问题。尽管大规模视觉语言模型如CLIP具备强大泛化能力，但其直接的图像-文本嵌入对齐存在固有信息不对称：图像同时包含类别语义和领域特定风格，而文本提示主要传递基础类别信息。这种不对称性阻碍了在真实场景下向新领域的泛化。为此，我们提出基于CLIP的风格归一化嵌入对齐方法SHED，不再对齐编码器的原始表示，而是对齐风格归一化后的嵌入。训练阶段，从各源域图像嵌入中移除领域特定风格中心，并对跨多种提示模板的文本嵌入平均后去除全局中心；推理时，因缺乏目标域信息，将多种文本领域中心投影至视觉空间，通过成员权重聚合预测。在五个基准上的大量实验表明，SHED性能显著优于现有方法（如在DomainNet上相比标准微调提升4.0%）。
原文摘要 · Abstract (English)
Domain generalization aims to enhance model robustness against unseen domains with embedding distribution shifts. While large-scale vision-language models like CLIP exhibit strong generalization, their direct image-text embedding alignment suffers from inherent information asymmetry: images encode both class semantics and domain-specific styles, whereas text prompts primarily convey basic class cues. This asymmetry hinders generalization to novel domains in realistic scenarios. To address this, we propose Style-Homogenized Embedding alignment for Domain-generalization (SHED), a novel CLIP-based method that aligns style-homogenized embeddings instead of raw representations from encoders in CLIP. During training, SHED removes domain-specific style centroids from both image embeddings computed per source domains and text embeddings which are averaged across diverse prompt templates and stripped of a global centroid. For inference, considering the lack of target domain information, SHED projects diverse textual domain centroids into the visual space and aggregates predictions via membership weighting. Extensive experiments on five benchmarks show SHED achieves state-of-the-art performance, outperforming prior methods significantly (e.g., +4.0\% on DomainNet vs. standard fine-tuning).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。