提出细粒度3D占位语义定位方法,提升自动驾驶感知精度
A Coarse-to-Fine Approach to Multi-Modality 3D Occupancy Grounding
- 从粗到精的多模态学习框架,融合视觉、语言与点云数据
- 在nuScenes上实现更精准的体素级占位预测,优于传统框标注
- 适用于需要高精度空间理解的自动驾驶场景
视觉定位旨在根据自然语言描述识别场景中的物体或区域,对自动驾驶的空间感知至关重要。然而,现有任务通常依赖边界框,难以捕捉细节——框内并非所有体素均被占据,导致表征不准确。为此,我们构建了一个面向复杂室外场景的3D占位语义定位基准,基于nuScenes数据集,融合自然语言与体素级占位标注,相比传统定位任务提供更高精度的物体感知。同时,提出GroundingOcc,一个端到端多模态模型,通过从粗到精的方式进行3D占位语义定位。该模型包含多模态编码器、占位头和定位头,并引入2D定位模块与深度估计模块以增强几何理解。大量实验表明,所提方法在该基准上显著优于现有基线。数据集已开源:https://github.com/RONINGOD/GroundingOcc。
原文摘要 · Abstract (English)
Visual grounding aims to identify objects or regions in a scene based on natural language descriptions, essential for spatially aware perception in autonomous driving. However, existing visual grounding tasks typically depend on bounding boxes that often fail to capture fine-grained details. Not all voxels within a bounding box are occupied, resulting in inaccurate object representations. To address this, we introduce a benchmark for 3D occupancy grounding in challenging outdoor scenes. Built on the nuScenes dataset, it integrates natural language with voxel-level occupancy annotations, offering more precise object perception compared to the traditional grounding task. Moreover, we propose GroundingOcc, an end-to-end model designed for 3D occupancy grounding through multi-modal learning. It combines visual, textual, and point cloud features to predict object location and occupancy information from coarse to fine. Specifically, GroundingOcc comprises a multimodal encoder for feature extraction, an occupancy head for voxel-wise predictions, and a grounding head to refine localization. Additionally, a 2D grounding module and a depth estimation module enhance geometric understanding, thereby boosting model performance. Extensive experiments on the benchmark demonstrate that our method outperforms existing baselines on 3D occupancy grounding. The dataset is available at https://github.com/RONINGOD/GroundingOcc.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。