针对全景分割中小物体漏检问题，提出新型遮罩变压器方法
PanSR: An Object-Centric Mask Transformer for Panoptic Segmentation
- 基于对象中心思想重构查询生成与监督机制
- 在LaRS上提升3.4点性能，显著改善小物体检测
- 适合自动驾驶等复杂场景下的视觉感知应用
全景分割是计算机视觉的基础任务，也是自动驾驶感知系统的关键组件。现有基于遮罩变压器的方法在标准基准上表现优异，但在小物体、密集场景和多尺度场景中面临挑战。我们发现当前方法存在三大根本缺陷：(i) 查询生成偏向大物体，导致小物体漏检；(ii) 初期定位准确的查询可能漂移到其他物体，造成漏检；(iii) 空间分离的实例可能合并为单一掩码，引发不一致的场景理解。为此，我们重新思考网络各组件及其监督方式，提出新型全景分割方法PanSR。该方法有效缓解实例合并问题，增强小物体检测能力，并在拥挤场景中提升性能，在具有挑战性的LaRS基准上相比最先进方法取得+3.4点的段级质量（PQ）提升，同时在Cityscapes上达到最先进水平。代码与模型将公开于https://github.com/lojzezust/PanSR。
原文摘要 · Abstract (English)
Panoptic segmentation is a fundamental task in computer vision and a crucial component for perception in autonomous vehicles. Recent mask-transformer-based methods achieve impressive performance on standard benchmarks but face significant challenges with small objects, crowded scenes and scenes exhibiting a wide range of object scales. We identify several fundamental shortcomings of the current approaches: (i) the query proposal generation process is biased towards larger objects, resulting in missed smaller objects, (ii) initially well-localized queries may drift to other objects, resulting in missed detections, (iii) spatially well-separated instances may be merged into a single mask causing inconsistent and false scene interpretations. To address these issues, we rethink the individual components of the network and its supervision, and propose a novel method for panoptic segmentation PanSR. PanSR effectively mitigates instance merging, enhances small-object detection and increases performance in crowded scenes, delivering a notable +3.4 PQ improvement over state-of-the-art on the challenging LaRS benchmark, while reaching state-of-the-art performance on Cityscapes. The code and models will be publicly available at https://github.com/lojzezust/PanSR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。