用证据学习让视觉变压器自动识别未知物体,无需事先知道异常数据。
Prior2Former -- Evidential Modeling of Mask Transformers for Assumption-Free Open-World Panoptic Segmentation

- 引入贝塔先验建模像素级不确定性,实现开集分割
- 在多个数据集上达到当前最优异常实例分割性能
- 无需异常样本或对比学习,适合真实场景部署
全景分割要求同类别中的个体实例可区分。现有先进方法依赖预定义类别,难以处理新类别和分布外(OOD)数据,这在自动驾驶等安全关键场景中尤为严重。本文提出 Prior2Former(P2F),首个基于证据学习的掩码视觉变压器方法,通过引入贝塔先验对像素级二值掩码分配中的模型不确定性进行建模。该设计实现了高质量的不确定性估计,能有效检测新类别和分布外对象,显著提升异常实例分割与开世界全景分割性能。与多数处理未知类别的分割模型不同,P2F无需访问分布外数据样本或对空类(即未标记类)进行对比训练,因此在实际应用中更具可行性。此外,P2F可灵活应用于异常实例分割与全景分割任务。在 Cityscapes、COCO、SegmentMeIfYouCan 与 OoDIS 数据集上的全面实验表明,P2F 在各项任务中均达到领先水平。
原文摘要 · Abstract (English)
In panoptic segmentation, individual instances must be separated within semantic classes. As state-of-the-art methods rely on a pre-defined set of classes, they struggle with novel categories and out-of-distribution (OOD) data. This is particularly problematic in safety-critical applications, such as autonomous driving, where reliability in unseen scenarios is essential. We address the gap between outstanding benchmark performance and reliability by proposing Prior2Former (P2F), the first approach for segmentation vision transformers rooted in evidential learning. P2F extends the mask vision transformer architecture by incorporating a Beta prior for computing model uncertainty in pixel-wise binary mask assignments. This design enables high-quality uncertainty estimation that effectively detects novel and OOD objects enabling state-of-the-art anomaly instance segmentation and open-world panoptic segmentation. Unlike most segmentation models addressing unknown classes, P2F operates without access to OOD data samples or contrastive training on void (i.e., unlabeled) classes, making it highly applicable in real-world scenarios where such prior information is unavailable. Additionally, P2F can be flexibly applied to anomaly instance and panoptic segmentation. Through comprehensive experiments on the Cityscapes, COCO, SegmentMeIfYouCan, and OoDIS datasets, P2F demonstrates state-of-the-art performance across the board.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。