通过全局场景与局部物体协同推理,提升3D点云中未知物体的检测准确率。
GLRD: Global-Local Collaborative Reason and Debate with PSL for 3D Open-Vocabulary Detection
- 融合局部物体与全局场景信息,用大模型进行常识推理优化检测
- 在ScanNet上提升3.72%、SUN RGB-D上提升14.11%的平均精度
- 适合做开放词汇3D目标检测的研究者和工业应用开发者
基于激光雷达的3D开放词汇检测(3D OVD)要求检测器在无现成标签的情况下从点云中识别新物体。以往方法仅关注物体级表征,忽略场景级信息,难以区分相似类别。本文提出全局-局部协同推理与概率软逻辑求解(GLRD)框架,结合大语言模型对物体与场景信息进行常识推理以优化检测结果。为提升决策精度,设计了面向开放词汇的概率软逻辑求解器(OV-PSL)与辩论机制,用于确认易混淆物体类别。针对类别分布不均问题,提出静态(SBC)与动态(DBC)平衡方案。为降低数据与训练噪声影响,引入反射伪标签生成(RPLG)与背景感知目标定位(BAOL)。在ScanNet和SUN RGB-D上的实验表明,该方法在部分开放词汇设置下,分别提升2.82%(SUN RGB-D)和3.72%(ScanNet)的平均精度;在全开放词汇设置下,分别提升4.03%(ScanNet)和14.11%(SUN RGB-D)。
原文摘要 · Abstract (English)
The task of LiDAR-based 3D Open-Vocabulary Detection (3D OVD) requires the detector to learn to detect novel objects from point clouds without off-the-shelf training labels. Previous methods focus on the learning of object-level representations and ignore the scene-level information, thus it is hard to distinguish objects with similar classes. In this work, we propose a Global-Local Collaborative Reason and Debate with PSL (GLRD) framework for the 3D OVD task, considering both local object-level information and global scene-level information. Specifically, LLM is utilized to perform common sense reasoning based on object-level and scene-level information, where the detection result is refined accordingly. To further boost the LLM's ability of precise decisions, we also design a probabilistic soft logic solver (OV-PSL) to search for the optimal solution, and a debate scheme to confirm the class of confusable objects. In addition, to alleviate the uneven distribution of classes, a static balance scheme (SBC) and a dynamic balance scheme (DBC) are designed. In addition, to reduce the influence of noise in data and training, we further propose Reflected Pseudo Labels Generation (RPLG) and Background-Aware Object Localization (BAOL). Extensive experiments conducted on ScanNet and SUN RGB-D demonstrate the superiority of GLRD, where absolute improvements in mean average precision are $+2.82\%$ on SUN RGB-D and $+3.72\%$ on ScanNet in the partial open-vocabulary setting. In the full open-vocabulary setting, the absolute improvements in mean average precision are $+4.03\%$ on ScanNet and $+14.11\%$ on SUN RGB-D.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。