通过显式建模视觉元素的空间关系提升跨域泛化能力
Domain Generalization through Spatial Relation Induction over Visual Primitives

- 将图像分解为视觉原语及其空间关系,显式学习结构组合
- 在CUB-DG上提升4.5个百分点,优于现有方法
- 适合需要强结构理解的跨域分类任务
领域泛化需识别在不同数据分布下仍稳定的表征。现有方法多通过优化训练过程(如模型选择、数据增强、特征对齐)实现稳定性,但隐含了结构组合的学习,限制了在组合性领域泛化基准上的表现。本文提出一种名为PARSE的图像分类框架,将视觉识别分解为视觉原语及其关系组成。通过在原语位置上使用软二元、三元和四元谓词表示组合,生成可端到端学习的空间对齐度量。架构包含三部分:(1)提取通用特征的CNN主干;(2)将特征映射为带可微坐标原语热图的概念瓶颈层;(3)评估检测到原语间候选空间关系的结构评分层。最终基于类别特异性关系组合的联合证据计算类别概率。在CUB-DG与DomainBed基准上,PARSE在CUB-DG上提升超4.5个百分点,且在DomainBed上保持竞争力。
原文摘要 · Abstract (English)
Domain generalization requires identifying stable representations that support reliable classification across domains. Most existing methods seek such stability through improving the training process, for example, through model selection strategies, data augmentation, or feature-alignment objectives. Although these strategies can be effective, they leave the representation learning of structural composition implicit, which may limit performance on compositional domain generalization benchmarks. In this work, we propose Primitive-Aware Relational Structure for domain gEneralization (PARSE), an image classification framework that factors visual recognition into visual primitives and their relational composition. We represent these compositions using soft binary, ternary, and quaternary predicates over primitive locations, yielding differentiable measures of spatial alignment that can be learned end-to-end. To learn primitives and relational structures jointly, we design an end-to-end architecture with three components: (1) a convolutional neural network (CNN) backbone that extracts general visual features, (2) a concept bottleneck layer that maps these features to primitive heatmaps with differentiable spatial coordinates, and (3) a structural scoring layer that evaluates candidate spatial relations among the detected primitives. We then compute class probability from the joint evidence of its class-specific relational compositions. Across CUB-DG and the DomainBed benchmark suite,PARSE improves accuracy by over 4.5 percentage points on CUB-DG and remains competitive with existing DG methods on DomainBed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。