将3D语义占据预测重定义为密集物体检测,提升实例级精度。
VoxDet: Rethinking 3D Semantic Occupancy Prediction as Dense Object Detection
- 把体素级标签转化为实例级偏移,实现无需训练的实例感知转换。
- 在SemanticKITTI上达到63.0的IoU,排名第一,兼顾高效与准确。
- 适合关注3D场景理解、实例分割与自动驾驶感知的研究者。
3D语义占据预测旨在重建周围环境的三维几何与语义信息。以往方法通常将其视为稠密分割任务,独立分类每个体素,但忽略了关键的实例级判别性,导致实例不完整和邻近模糊。本文指出:体素级类别标签隐含了实例级信息,这一“免费午餐”被社区忽视。为此,我们提出无需训练的体素到实例转换(VoxNT)技巧,将体素标签自动转为实例偏移标签。在此基础上,提出VoxDet框架,将体素级占据预测重构为密集物体检测,分解为偏移回归与语义预测两个子任务。首先,通过空间解耦体素编码器生成解耦特征体,学习稠密投影三重感知空间中的任务特定形变;随后,采用任务解耦密集预测器,回归4维偏移场以估计体素到物体边界的6个方向距离,再用该偏移引导分类分支的实例级聚合,实现实例感知预测。实验表明,VoxDet可适配相机与激光雷达输入,在双基准测试上均达当前最优性能,且效率高,于SemanticKITTI测试集上取得63.0 IoU,位居在线排行榜第一。
原文摘要 · Abstract (English)
3D semantic occupancy prediction aims to reconstruct the 3D geometry and semantics of the surrounding environment. With dense voxel labels, prior works typically formulate it as a dense segmentation task, independently classifying each voxel. However, this paradigm neglects critical instance-centric discriminability, leading to instance-level incompleteness and adjacent ambiguities. To address this, we highlight a free lunch of occupancy labels: the voxel-level class label implicitly provides insight at the instance level, which is overlooked by the community. Motivated by this observation, we first introduce a training-free Voxel-to-Instance (VoxNT) trick: a simple yet effective method that freely converts voxel-level class labels into instance-level offset labels. Building on this, we further propose VoxDet, an instance-centric framework that reformulates the voxel-level occupancy prediction as dense object detection by decoupling it into two sub-tasks: offset regression and semantic prediction. Specifically, based on the lifted 3D volume, VoxDet first uses (a) Spatially-decoupled Voxel Encoder to generate disentangled feature volumes for the two sub-tasks, which learn task-specific spatial deformation in the densely projected tri-perceptive space. Then, we deploy (b) Task-decoupled Dense Predictor to address this task via dense detection. Here, we first regress a 4D offset field to estimate distances (6 directions) between voxels and object borders in the voxel space. The regressed offsets are then used to guide the instance-level aggregation in the classification branch, achieving instance-aware prediction. Experiments show that VoxDet can be deployed on both camera and LiDAR input, jointly achieving state-of-the-art results on both benchmarks. VoxDet is not only highly efficient, but also achieves 63.0 IoU on the SemanticKITTI test set, ranking 1st on the online leaderboard.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。