arXiv:2412.12740cs.CVcs.RO2024-12被引 1

提出新方法与数据集,让自动驾驶系统实时识别未知物体。

Open-World Panoptic Segmentation

  • 基于增量学习框架,动态发现新类别和新实例
  • 在800张图像上识别出50+未知类别、4000个实例
  • 专为自动驾驶设计,适合研究开放世界感知的团队

感知是自主视觉系统(如自动驾驶汽车)的关键组件。系统需在真实环境中理解周围环境以安全运行,且必须能应对从未见过的新物体和新场景。本文提出开放世界全景分割任务,即在测试时发现新的语义类别和对象实例,同时保持已发现类别的统一性。我们提出Con2MAV方法,扩展了先前用于开放世界语义分割的ContMAV模型。在多个数据集上的实验表明,该模型在开放世界分割任务中达到当前最优性能,同时对已知类别仍保持良好表现。我们还构建了PANIC(Panoptic ANomalies In Context)基准数据集,用于评估自动驾驶场景下的开放世界全景分割。该数据集由车载多模态传感器采集,包含800张图像,超过50个训练集未见的未知类别及4000个物体实例,具有高精度像素级标注。我们将在隐藏测试集上举办竞赛,代码与数据集已公开于https://www.ipb.uni-bonn.de/data/panic。

原文摘要 · Abstract (English)

Perception is a key building block of autonomously acting vision systems such as autonomous vehicles. It is crucial that these systems are able to understand their surroundings in order to operate safely and robustly. Additionally, autonomous systems deployed in unconstrained real-world scenarios must be able of dealing with novel situations and object that have never been seen before. In this article, we tackle the problem of open-world panoptic segmentation, i.e., the task of discovering new semantic categories and new object instances at test time, while enforcing consistency among the categories that we incrementally discover. We propose Con2MAV, an approach for open-world panoptic segmentation that extends our previous work, ContMAV, which was developed for open-world semantic segmentation. Through extensive experiments across multiple datasets, we show that our model achieves state-of-the-art results on open-world segmentation tasks, while still performing competitively on the known categories. We will open-source our implementation upon acceptance. Additionally, we propose PANIC (Panoptic ANomalies In Context), a benchmark for evaluating open-world panoptic segmentation in autonomous driving scenarios. This dataset, recorded with a multi-modal sensor suite mounted on a car, provides high-quality, pixel-wise annotations of anomalous objects at both semantic and instance level. Our dataset contains 800 images, with more than 50 unknown classes, i.e., classes that do not appear in the training set, and 4000 object instances, making it an extremely challenging dataset for open-world segmentation tasks in the autonomous driving scenario. We provide competitions for multiple open-world tasks on a hidden test set. Our dataset and competitions are available at https://www.ipb.uni-bonn.de/data/panic.

开放世界全景分割自动驾驶增量学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。