将3D高斯点云引入室内三维目标检测,提升物体与背景区分度。
3DGS-DET: Empower 3D Gaussian Splatting with Boundary Guidance and Box-Focused Sampling for Indoor 3D Object Detection
- 用2D边界信息引导高斯点分布,增强3D空间结构清晰度
- 基于2D框生成概率分布,减少背景噪声点,保留更多物体点
- 在ScanNet和ARKITScenes上显著超越现有方法,最高提升14.9%
神经辐射场(NeRF)已被用于室内三维目标检测(3DOD),通过视图合成表示提供有前景的方案,但其隐式表示限制了表达能力。最近,3D高斯点云(3DGS)作为一种显式3D表示方法应运而生,克服了这一局限。本文首次将3DGS引入室内3DOD,识别出两大挑战:(i) 高斯点云的空间分布模糊——3DGS主要依赖2D像素级监督,导致高斯点在3D空间中分布不清,难以区分物体与背景;(ii) 过多背景点——2D图像包含大量背景像素,导致3DGS重建密集且包含大量表示背景的噪声点,影响检测性能。针对挑战(i),我们利用3DGS从2D图像生成的特性,提出2D边界引导机制,显著优化高斯点的空间分布,实现物体与背景更清晰的区分。针对挑战(ii),提出框聚焦采样策略,通过2D检测框生成3D空间中的物体概率分布,实现高效的概率采样,保留更多物体点并抑制背景噪声。得益于这些创新,3DGS-DET显著优于当前最先进的基于NeRF的方法NeRF-Det++,在ScanNet上分别提升+6.0([email protected])和+7.8([email protected]),在ARKITScenes上提升+14.9([email protected])。
原文摘要 · Abstract (English)
Neural Radiance Fields (NeRF) have been adapted for indoor 3D Object Detection (3DOD), offering a promising approach to indoor 3DOD via view-synthesis representation. But its implicit nature limits representational capacity. Recently, 3D Gaussian Splatting (3DGS) has emerged as an explicit 3D representation that addresses the limitation. This work introduces 3DGS into indoor 3DOD for the first time, identifying two main challenges: (i) Ambiguous spatial distribution of Gaussian blobs -- 3DGS primarily relies on 2D pixel-level supervision, resulting in unclear 3D spatial distribution of Gaussian blobs and poor differentiation between objects and background, which hinders indoor 3DOD; (ii) Excessive background blobs -- 2D images typically include numerous background pixels, leading to densely reconstructed 3DGS with many noisy Gaussian blobs representing the background, negatively affecting detection. To tackle (i), we leverage the fact that 3DGS reconstruction is derived from 2D images, and propose an elegant solution by incorporating 2D Boundary Guidance to significantly enhance the spatial distribution of Gaussian blobs, resulting in clearer differentiation between objects and their background (please see fig:teaser). To address (ii), we propose a Box-Focused Sampling strategy using 2D boxes to generate object probability distribution in 3D space, allowing effective probabilistic sampling in 3D to retain more object blobs and reduce noisy background blobs. Benefiting from these innovations, 3DGS-DET significantly outperforms the state-of-the-art NeRF-based method, NeRF-Det++, achieving improvements of +6.0 on [email protected] and +7.8 on [email protected] for the ScanNet, and the +14.9 on [email protected] for the ARKITScenes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。