让3D检测识别未知物体及其属性，无需预先知道类别尺寸
Towards Open-Vocabulary Multimodal 3D Object Detection with Attributes
- 用基础模型融合3D特征与文本，联合检测物体和属性
- 在nuScenes和Argoverse 2上超越现有方法，识别新类别与属性
- 构建新数据集OVAD，含丰富属性标注，支持开放词汇研究
3D目标检测在自动驾驶系统中至关重要，但现有方法受限于封闭集假设，在真实场景中难以识别新物体及其属性。本文提出OVODA框架，实现无需已知新类别锚框尺寸的开放词汇3D物体与属性检测。该框架利用基础模型弥合3D特征与文本间的语义鸿沟，同时检测空间关系、运动状态等属性。为推动此方向，我们构建了新数据集OVAD，补充现有3D检测基准的属性标注。OVODA引入多项创新：基础模型特征拼接、提示调优策略，以及针对属性检测的视角特定提示与水平翻转增强。在nuScenes和Argoverse 2数据集上的实验表明，当未提供新类别锚框尺寸时，OVODA在开放词汇3D检测任务中优于现有最先进方法，并成功识别物体属性。数据集已公开：https://doi.org/10.5281/zenodo.16904069。
原文摘要 · Abstract (English)
3D object detection plays a crucial role in autonomous systems, yet existing methods are limited by closed-set assumptions and struggle to recognize novel objects and their attributes in real-world scenarios. We propose OVODA, a novel framework enabling both open-vocabulary 3D object and attribute detection with no need to know the novel class anchor size. OVODA uses foundation models to bridge the semantic gap between 3D features and texts while jointly detecting attributes, e.g., spatial relationships, motion states, etc. To facilitate such research direction, we propose OVAD, a new dataset that supplements existing 3D object detection benchmarks with comprehensive attribute annotations. OVODA incorporates several key innovations, including foundation model feature concatenation, prompt tuning strategies, and specialized techniques for attribute detection, including perspective-specified prompts and horizontal flip augmentation. Our results on both the nuScenes and Argoverse 2 datasets show that under the condition of no given anchor sizes of novel classes, OVODA outperforms the state-of-the-art methods in open-vocabulary 3D object detection while successfully recognizing object attributes. Our OVAD dataset is released here: https://doi.org/10.5281/zenodo.16904069 .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。