让模型关注局部不变特征,测试时还能自动修正预测结果。
TIDE: Training Locally Interpretable Domain Generalization Models Enables Test-time Correction
- 训练时强制模型关注局部概念,通过注意力对齐和对比学习提升鲁棒性。
- 在四个基准数据集上平均性能超越当前最优水平12%。
- 可生成可视化注意力图,适合需要可解释性的工业场景应用。
我们研究单源域泛化问题。现有方法依赖大量数据增强来覆盖多样域,但难以应对语义偏移(如背景、视角变化),因其常学习全局特征而非具有域不变性的局部概念。为此,我们提出一种新方法,使模型在预测时利用这些局部概念。由于缺乏带有类别级概念和定位图的标注数据集,我们首先构建了一条新流水线,利用扩散模型和大语言模型的丰富特征生成标注。我们的第二个创新是TIDE训练方案,包含概念显著性对齐损失,确保模型聚焦正确概念区域;以及局部概念对比损失,促进学习域不变的概念表征。这不仅提升了模型鲁棒性,还可通过预测的概念显著性图实现可视化解释。测试时,我们进一步提出一种校正算法,利用对应局部概念表示,迭代优化预测直至与训练结束时存储的原型概念表示对齐。我们在四个标准域泛化基准数据集上进行了全面评估,性能显著优于当前最先进方法(平均提升12%),同时验证了预测结果的可解释性。
原文摘要 · Abstract (English)
We consider the problem of single-source domain generalization. Existing methods typically rely on extensive augmentations to synthetically cover diverse domains during training. However, they struggle with semantic shifts (e.g., background and viewpoint changes), as they often learn global features instead of local concepts that tend to be domain invariant. To address this gap, we propose an approach that compels models to leverage such local concepts during prediction. Given no suitable dataset with per-class concepts and localization maps exists, we first develop a novel pipeline to generate annotations by exploiting the rich features of diffusion and large-language models. Our next innovation is TIDE, a novel training scheme with a concept saliency alignment loss that ensures model focus on the right per-concept regions and a local concept contrastive loss that promotes learning domain-invariant concept representations. This not only gives a robust model but also can be visually interpreted using the predicted concept saliency maps. Given these maps at test time, our final contribution is a new correction algorithm that uses the corresponding local concept representations to iteratively refine the prediction until it aligns with prototypical concept representations that we store at the end of model training. We evaluate our approach extensively on four standard DG benchmark datasets and substantially outperform the current state-ofthe-art (12% improvement on average) while also demonstrating that our predictions can be visually interpreted
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。