首次实现单目3D开放集目标检测，提升真实场景适应能力。
3D-MOOD: Lifting 2D to 3D for Monocular Open-Set Object Detection
- 将2D开放集检测拓展至3D空间，设计端到端联合训练的3D框头。
- 在Omni3D和ScanNet数据集上超越现有方法，实现新基准性能。
- 适用于机器人、AR/VR等需应对未知环境与物体的真实场景。
单目3D目标检测在机器人、AR/VR等领域具有重要价值。现有方法受限于封闭集设定，即训练与测试数据来自相同场景或类别。但真实应用常面临新环境与新型物体，对模型泛化能力提出挑战。本文首次提出端到端的单目3D开放集目标检测器（3D-MOOD），通过设计3D边界框头，将2D开放集检测提升至3D空间，实现2D与3D任务的联合训练，显著提升整体性能。通过引入几何先验条件化对象查询，增强跨场景3D估计的泛化能力。为进一步提升效果，设计了规范图像空间以支持更高效的跨数据集训练。在封闭集（Omni3D）与开放集（Omni3D→Argoverse 2、ScanNet）设置下均取得最新最佳结果。代码与模型见 royyang0714.github.io/3D-MOOD。
原文摘要 · Abstract (English)
Monocular 3D object detection is valuable for various applications such as robotics and AR/VR. Existing methods are confined to closed-set settings, where the training and testing sets consist of the same scenes and/or object categories. However, real-world applications often introduce new environments and novel object categories, posing a challenge to these methods. In this paper, we address monocular 3D object detection in an open-set setting and introduce the first end-to-end 3D Monocular Open-set Object Detector (3D-MOOD). We propose to lift the open-set 2D detection into 3D space through our designed 3D bounding box head, enabling end-to-end joint training for both 2D and 3D tasks to yield better overall performance. We condition the object queries with geometry prior and overcome the generalization for 3D estimation across diverse scenes. To further improve performance, we design the canonical image space for more efficient cross-dataset training. We evaluate 3D-MOOD on both closed-set settings (Omni3D) and open-set settings (Omni3D to Argoverse 2, ScanNet), and achieve new state-of-the-art results. Code and models are available at royyang0714.github.io/3D-MOOD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。