解决全景分割在跨域场景下的视角扭曲与未知类别问题。
Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentation
- 提出Euler-Margin Attention,用角度边界增强视角不变特征
- 在四个数据集上实现当前最优性能,对新类别泛化强
- 适合需要跨场景、跨视角全景理解的自动驾驶等应用
跨域全景语义分割因能支持真实世界中360°场景理解而受到关注。然而,严重的视场(FoV)畸变和跨域不一致的开放集语义使其极具挑战性。本文提出开放集域适应设置,并设计可外推的域自适应全景分割框架(EDA-PSeg),该框架在局部透视图像上训练,测试时处理完整的360°全景图,显式应对跨域几何视场偏移与未见类别的语义不确定性。提出欧拉-边距注意力(EMA),引入角度边界以增强视角不变的语义表征,同时进行幅度与相位调制以提升对未见类别的泛化能力。设计图匹配适配器(GMA),构建高阶图关系,对齐跨视场共享语义,并通过结构自适应有效分离新类别。在四个基准数据集上,针对相机位移、天气变化及开放集场景的大量实验表明,EDA-PSeg在多种视图几何条件下表现卓越,具备强鲁棒性与环境适应力。代码已开源。
原文摘要 · Abstract (English)
Cross-domain panoramic semantic segmentation has attracted growing interest as it enables comprehensive 360° scene understanding for real-world applications. However, it remains particularly challenging due to severe geometric Field of View (FoV) distortions and inconsistent open-set semantics across domains. In this work, we formulate an open-set domain adaptation setting, and propose Extrapolative Domain Adaptive Panoramic Segmentation (EDA-PSeg) framework that trains on local perspective views and tests on full 360° panoramic images, explicitly tackling both geometric FoV shifts across domains and semantic uncertainty arising from previously unseen classes. To this end, we propose the Euler-Margin Attention (EMA), which introduces an angular margin to enhance viewpoint-invariant semantic representation, while performing amplitude and phase modulation to improve generalization toward unseen classes. Additionally, we design the Graph Matching Adapter (GMA), which builds high-order graph relations to align shared semantics across FoV shifts while effectively separating novel categories through structural adaptation. Extensive experiments on four benchmark datasets under camera-shift, weather-condition, and open-set scenarios demonstrate that EDA-PSeg achieves state-of-the-art performance, robust generalization to diverse viewing geometries, and resilience under varying environmental conditions. The code is available at https://github.com/zyfone/EDA-PSeg.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。