无需训练，用大模型直接识别图像中未知物体区域。
Training-Free Out-Of-Distribution Segmentation With Foundation Models
- 用InternImage特征+K-Means聚类与置信度阈值，无监督划分异常区域。
- 在RoadAnomaly和ADE-OoD数据集上分别达50.02和48.77平均精度。
- 适合追求零样本、低依赖的异常分割应用开发者使用。
在自动驾驶等安全关键场景中，检测语义分割中的未知物体至关重要。大型视觉基础模型（如DINOv2、InternImage、CLIP）通过提供泛化能力强的特征，推动了视觉表征学习的发展。尽管其在闭集语义任务中表现优异，但对分割任务中分布外（OoD）区域的检测能力仍待探索。本文研究基础模型在分割数据集微调后，是否能在无异常标注的情况下自然区分分布内（ID）与分布外（OoD）区域。提出一种简单、无需训练的方法：利用InternImage主干提取特征，结合K-Means聚类与原始解码器输出置信度阈值，识别分布外聚类。该方法在RoadAnomaly基准上达到50.02平均精度，在ADE-OoD基准上达48.77，优于多个有监督与无监督基线。结果表明，该方向为所需假设少、无需额外数据的通用分布外分割提供了前景。
原文摘要 · Abstract (English)
Detecting unknown objects in semantic segmentation is crucial for safety-critical applications such as autonomous driving. Large vision foundation models, including DINOv2, InternImage, and CLIP, have advanced visual representation learning by providing rich features that generalize well across diverse tasks. While their strength in closed-set semantic tasks is established, their capability to detect out-of-distribution (OoD) regions in semantic segmentation remains underexplored. In this work, we investigate whether foundation models fine-tuned on segmentation datasets can inherently distinguish in-distribution (ID) from OoD regions without any outlier supervision. We propose a simple, training-free approach that utilizes features from the InternImage backbone and applies K-Means clustering alongside confidence thresholding on raw decoder logits to identify OoD clusters. Our method achieves 50.02 Average Precision on the RoadAnomaly benchmark and 48.77 on the benchmark of ADE-OoD with InternImage-L, surpassing several supervised and unsupervised baselines. These results suggest a promising direction for generic OoD segmentation methods that require minimal assumptions or additional data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。