提升3D物体检测中未见物体的发现与训练鲁棒性，实现更准更稳的开放词汇识别。
Open-Vocabulary 3D Object Detection with Co-Distillation Discovery and Dual Guidance Robust Training
- 通过几何、结构与语义综合得分的匈牙利匹配，实现高质量新物体发现。
- 在SUN RGB-D和ScanNetV2上显著超越现有方法，性能提升明显。
- 适合关注开放词汇3D检测、数据不完整场景下的模型泛化研究者。
近年来，开放词汇3D物体检测（3D-OVD）因其能识别3D场景中未见物体而受到广泛关注。现有方法通常采用两阶段流程：先用基础模型发现新物体，再基于发现结果训练3D-OVD模型。然而，该流程在发现阶段常出现定位不准与分类不匹配问题，进而制约训练阶段性能。为此，本文提出一种新框架，旨在提升新物体发现的可靠性与模型训练的鲁棒性。针对可靠发现，提出共蒸馏策略，通过匈牙利匹配综合几何一致性、结构合理性与语义确定性得分，提取高质量新物体。为增强训练鲁棒性，设计双引导学习机制：回归头引入场景感知不确定性正则化，分类头采用大语言模型引导的层次对齐，有效缓解不精确3D边界框与语义模糊带来的负面影响。在SUN RGB-D与ScanNetV2上的大量实验表明，本方法显著优于现有最先进方法。代码已公开于https://github.com/shangboyuan/Co-3DGT。
原文摘要 · Abstract (English)
Recently, open-vocabulary 3D object detection (3D-OVD) has gained increasing attention for its ability to detect unseen objects in 3D scenes. Existing approaches typically adopt a two-stage pipeline that first discovers novel objects using foundation models and then trains a 3D-OVD model based on these discovered objects. Although effective, this pipeline often suffers from inaccurate localization and mismatched classification during the discovery stage, which subsequently limits the performance of the model training stage. To address these limitations, we advocate for improving both the reliability of novel object discovery and the robustness of model training, and propose an innovative framework. Specifically, for reliable discovery, our co-distillation strategy distills high-quality novel objects by applying Hungarian matching over a comprehensive score that incorporates geometric consistency, structural objectness, and semantic certainty. To enhance robust model training, we further propose a dual-guidance learning scheme, incorporating a scene-awareness-guided uncertainty regularization for the regression head and an LLM-guided hierarchical alignment for the classification head, effectively mitigating the negative effects of imprecise 3D bounding boxes and semantic ambiguity. Extensive experiments on SUN RGB-D and ScanNetV2 demonstrate that our method achieves significant performance gains over state-of-the-art approaches. Code is available at https://github.com/shangboyuan/Co-3DGT
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。