提出新评估框架,区分近、远、更远的未知目标检测,更真实反映实际场景
FindMeIfYouCan: Bringing Open Set metrics to $\textit{near} $, $ \textit{far} $ and $\textit{farther}$ Out-of-Distribution Object Detection
- 按语义和视觉距离划分未知目标为近/远/更远三类,构建新评估基准
- 发现越接近已知类别的未知目标越易定位但越易被误判为已知类
- 引入开放集评测指标,帮助识别模型对未知目标的忽视与误判行为
当前主流目标检测方法多基于封闭世界假设,即测试时类别与训练一致。但在自动驾驶、医学影像等安全关键领域,识别和定位未知物体至关重要。近年来,分布外(OOD)检测成为重要研究方向,聚焦于识别与未知物体相关的错误预测。本文指出,现有OOD目标检测评估协议违反了与已知分布(ID)对象不重叠的假设,掩盖了忽略未知物体等关键情形,可能导致部署时过度自信。为此,我们手动构建并扩充基准数据集,利用语义相似性创建分为‘近’、‘远’、‘更远’三类的新评估划分。同时引入开放集学习中的成熟度量指标,深入分析方法在检测未知目标、忽略未知目标以及将OOD误判为ID方面的表现。全面评估表明:语义和视觉上接近已知分布的未知目标较易定位,但也更容易被误认为已知类别;而‘远’和‘更远’的未知目标更难定位,但较少被误判为已知类。
原文摘要 · Abstract (English)
State-of-the-art Object Detection (OD) methods predominantly operate under a closed-world assumption, where test-time categories match those encountered during training. However, detecting and localizing unknown objects is crucial for safety-critical applications in domains such as autonomous driving and medical imaging. Recently, Out-Of-Distribution (OOD) detection has emerged as a vital research direction for OD, focusing on identifying incorrect predictions typically associated with unknown objects. This paper shows that the current evaluation protocol for OOD-OD violates the assumption of non-overlapping objects with respect to the In-Distribution (ID) datasets, and obscures crucial situations such as ignoring unknown objects, potentially leading to overconfidence in deployment scenarios where truly novel objects might be encountered. To address these limitations, we manually curate, and enrich the existing benchmark by exploiting semantic similarity to create new evaluation splits categorized as $\textit{near}$, $\textit{far}$, and $\textit{farther}$ from ID distributions. Additionally, we incorporate established metrics from the Open Set community, providing deeper insights into how effectively methods detect unknowns, when they ignore them, and when they mistakenly classify OOD objects as ID. Our comprehensive evaluation demonstrates that semantically and visually close OOD objects are easier to localize than far ones, but are also more easily confounded with ID objects. $\textit{Far}$ and $\textit{farther}$ objects are harder to localize but less prone to be taken for an ID object.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。