arXiv:2607.18112cs.CV2026-07中稿 · the 2026 IEEE Inte…

通过注意力机制融合遮挡信息,提升复杂场景下全景分割的准确性。

Occlusion-Aware Panoptic Segmentation with Joint Position Embedding and Occlusion-Level Attention

论文配图:Occlusion-Aware Panoptic Segmentation with Joint Position Embedding and Occlusion-Level Attention
图 1 · 摘自论文原文
  • 用遮挡等级注意力与位置嵌入联合调制,增强模型对遮挡的感知能力。
  • 在COCO-OLAC和Cityscapes-OLAC上均实现性能提升,计算开销极小。
  • 适用于需要精准处理遮挡的视觉任务,如自动驾驶、机器人感知。

复杂场景中的全景分割仍受遮挡影响,但现有方法常忽略遮挡建模。本文提出位置嵌入调制与遮挡等级注意力(PEMOLA)模块,可无缝集成于基于Transformer的全景分割框架中。通过在COCO-OLAC数据集上训练遮挡分类器,获取遮挡等级注意力作为空间引导,并将遮挡标签编码为可学习嵌入,生成通道权重。通过联合调制,将遮挡先验融入位置嵌入,显著提升遮挡建模能力。进一步按相同标注协议构建了城市景观遮挡标签数据集(Cityscapes-OLAC),用于评估跨数据集泛化性。大量实验表明,PEMOLA在COCO-OLAC和Cityscapes-OLAC上持续提升全景分割性能,且计算开销微小。结果凸显了遮挡建模的重要性,引入遮挡等级注意力有助于在遮挡条件下实现鲁棒的全景分割。代码与数据集见https://github.com/wenbo-wei/PEMOLA。

原文摘要 · Abstract (English)

Panoptic segmentation in complex scenes remains challenging because of occlusions, yet modern approaches often neglect occlusion modelling. In this paper, we propose Position Embedding Modulation with Occlusion Level Attention (PEMOLA), a novel occlusion-aware module that can be seamlessly integrated into transformer-based panoptic segmentation. To obtain occlusion cues, we train an occlusion classifier on the COCO-OLAC dataset. The classifier derives the occlusion-level attention, which serves as spatial guidance, while the occlusion labels are encoded into a learnable embedding to produce channel-wise weights. Through joint modulation, PEMOLA elegantly introduces the occlusion priors into the position embedding, thereby improving the occlusion modelling. We further annotate the Cityscapes dataset with occlusion levels, termed Cityscapes Occlusion Labels for All Computer Vision Tasks (Cityscapes-OLAC), following the same labelling protocol as COCO-OLAC, to evaluate the cross-dataset generalisation ability of PEMOLA. Extensive experiments on COCO-OLAC and Cityscapes-OLAC demonstrate that PEMOLA consistently improves panoptic segmentation quality while introducing minimal computational overhead. These results highlight the importance of occlusion modelling, where incorporating occlusion-level attention helps deliver robust panoptic segmentation under occlusion. Code and dataset are available at https://github.com/wenbo-wei/PEMOLA.

全景分割遮挡建模Transformer注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。