通过通道级稀疏化学习跨域不变特征,提升模型泛化能力
Learning domain-invariant features through channel-level sparsification for Out-Of Distribution Generalization
- 用通道级因果掩码实现特征稀疏,分离因果与伪相关特征
- 在多个OOD基准上超越现有顶尖方法,显著提升泛化性能
- 适合需要强鲁棒性的图像分析系统,如医疗影像识别
分布外(OOD)泛化已成为评估图像分析系统的关键指标。由于深度学习模型常捕捉域特定上下文,易形成对非因果特征的捷径依赖,导致在不同数据源上表现不一致。现有方法如不变性学习虽能缓解此问题,但在深层潜在空间中难以分离高度混合的特征,无法彻底解决捷径学习问题。本文提出层级因果丢弃(HCD),通过通道级因果掩码强制特征稀疏,使模型在表示层实现因果干预,有效区分因果特征与伪特征。训练采用基于矩阵的互信息(MMI)目标,最小化潜在特征与域标签之间的互信息,同时最大化与类别标签共享的信息。为保证稳定性,引入基于StyleMix的VICReg模块,防止掩码误过滤关键因果信息。在多个OOD基准上的实验表明,HCD优于现有顶级方法。
原文摘要 · Abstract (English)
Out-of-Distribution (OOD) generalization has become a primary metric for evaluating image analysis systems. Since deep learning models tend to capture domain-specific context, they often develop shortcut dependencies on these non-causal features, leading to inconsistent performance across different data sources. Current techniques, such as invariance learning, attempt to mitigate this. However, they struggle to isolate highly mixed features within deep latent spaces. This limitation prevents them from fully resolving the shortcut learning problem.In this paper, we propose Hierarchical Causal Dropout (HCD), a method that uses channel-level causal masks to enforce feature sparsity. This approach allows the model to separate causal features from spurious ones, effectively performing a causal intervention at the representation level. The training is guided by a Matrix-based Mutual Information (MMI) objective to minimize the mutual information between latent features and domain labels, while simultaneously maximizing the information shared with class labels.To ensure stability, we incorporate a StyleMix-driven VICReg module, which prevents the masks from accidentally filtering out essential causal data. Experimental results on OOD benchmarks show that HCD performs better than existing top-tier methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。