用文本驱动因果学习,提升无源域泛化能力
Text-Driven Causal Representation Learning for Source-Free Domain Generalization
- 通过文本提示生成风格向量,模拟视觉表征
- 在多个数据集上达到领先性能,最高提升3.2%准确率
- 适合缺乏源域数据的现实场景,如医疗图像分析
深度学习在训练与测试分布不一致时表现不佳。传统域泛化需多源域数据,成本高昂。近期视觉-语言模型如CLIP可通过文本提示实现无源域泛化(SFDG),降低数据需求。但现有方法难以处理域特异性混淆因子,限制泛化能力。为此,我们提出TDCRL(文本驱动因果表示学习),首个将因果推断引入SFDG的方法。TDCRL分两步:首先利用数据增强生成风格词向量,结合类别信息生成文本嵌入以模拟视觉表征;其次训练因果干预网络,配合混淆因子字典提取域不变特征。基于因果学习,该方法提供清晰有效的机制,实现鲁棒域不变特征,确保强泛化性。在PACS、VLCS、OfficeHome和DomainNet上的大量实验表明,TDCRL达到当前最优性能,验证了其在SFDG中的有效性。
原文摘要 · Abstract (English)
Deep learning often struggles when training and test data distributions differ. Traditional domain generalization (DG) tackles this by including data from multiple source domains, which is impractical due to expensive data collection and annotation. Recent vision-language models like CLIP enable source-free domain generalization (SFDG) by using text prompts to simulate visual representations, reducing data demands. However, existing SFDG methods struggle with domain-specific confounders, limiting their generalization capabilities. To address this issue, we propose TDCRL (\textbf{T}ext-\textbf{D}riven \textbf{C}ausal \textbf{R}epresentation \textbf{L}earning), the first method to integrate causal inference into the SFDG setting. TDCRL operates in two steps: first, it employs data augmentation to generate style word vectors, combining them with class information to generate text embeddings to simulate visual representations; second, it trains a causal intervention network with a confounder dictionary to extract domain-invariant features. Grounded in causal learning, our approach offers a clear and effective mechanism to achieve robust, domain-invariant features, ensuring robust generalization. Extensive experiments on PACS, VLCS, OfficeHome, and DomainNet show state-of-the-art performance, proving TDCRL effectiveness in SFDG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。