arXiv:2608.08308cs.CVcs.AI2026-08中稿 · the 2026 IEEE Inte…

提出三路解码器,让模型既能分清已知类,又能发现未知异常。

Open-World Semantic Segmentation with Sensitivity Modeling

论文配图:Open-World Semantic Segmentation with Sensitivity Modeling
图 1 · 摘自论文原文
  • 用三个互补解码器分别捕捉类别差异、特征能量和激活不稳定性
  • 在BDD-Anomaly上异常检测AUROC提升2.4%,误报率降低2.5个百分点
  • 适合需要识别未知内容的自动驾驶等开放世界场景

现代视觉系统需在开放世界中运行,既要识别已知类别,又要检测未见或异常内容。传统语义分割模型基于封闭世界假设,对新内容常产生过度自信的错误分类。本文提出开放世界语义分割方法,联合完成已知类分割与未知/异常内容检测,无需额外标注。在统一编码器-解码器架构中引入第三个互补解码器——敏感性解码器,捕捉细粒度纹理异常与激活不稳定性,这些是语义不确定性的重要信号,而常规原型或对比学习难以察觉。三个解码器分别提供:日志空间中的类别级离群距离、嵌入空间中的全局特征能量、跨编码器尺度的局部激活不稳定性。在Cityscapes与BDD-Anomaly数据集上的实验表明,该方法在保持良好闭集精度的同时,显著提升异常分割与新类别发现能力,在BDD-Anomaly上实现+2.4% AUROC与2.5 pp. FPR@95TPR下降,优于基线模型。

原文摘要 · Abstract (English)

Modern vision systems must operate in "open-world" settings, where models must recognize known categories and detect unseen or anomalous content. Conventional semantic segmentation models operate under a "closed-world" assumption, often producing overconfident misclassifications on novel content. We address open-world semantic segmentation, the joint task of segmenting known classes while detecting and grouping novel or anomalous content without additional supervision, by extending a dual-decoder baseline with a third, complementary decoder within a unified encoder-decoder design. The first decoder performs closed-set segmentation using Gaussian prototypes for known categories. The second uses contrastive feature learning to isolate unknown regions in embedding space. The third, our key contribution, is a sensitivity decoder that captures fine-grained texture irregularities and activation instabilities indicative of semantic uncertainty, which neither semantic prototypes nor contrastive norms can reliably detect. The three decoders provide genuinely complementary signals: class-level OOD distance in logit space, global feature energy in embedding space, and local activation instability across encoder scales. Experiments on Cityscapes and BDD-Anomaly show that our method improves anomaly segmentation and novel-class discovery while maintaining competitive closed-set accuracy, with gains of +2.4% AUROC and a 2.5 pp. reduction in FPR@95TPR on BDD-Anomaly over the baseline.

开放世界异常检测语义分割感知不确定性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。