arXiv:2602.03371cs.CV2026-02中稿 · TIP 2026被引 2

解决摄像头3D语义场景补全中的稀疏体素问题,提升感知精度。

Multi-Resolution Alignment for Voxel Sparsity in Camera-Based 3D Semantic Scene Completion

  • 通过多分辨率特征对齐,利用场景和实例级信息辅助监督。
  • 在nuScenes数据集上,语义类别平均交并比提升至52.6%,优于基线方法。
  • 适合需要高精度3D环境理解的自动驾驶系统研发人员使用。

基于摄像头的3D语义场景补全(SSC)通过图像输入实现对周围3D场景中每个体素的几何占据与语义标签评估,为感知-预测-规划一体化的自动驾驶系统提供体素级场景理解基础。尽管现有方法取得显著进展,但其优化仅依赖体素标签的监督,在自动驾驶场景中大量体素为空,导致体素稀疏问题,制约了优化效率与模型性能。为此,本文提出多分辨率对齐(MRA)方法,通过跨多分辨率3D特征的场景与实例级对齐,缓解体素稀疏问题。具体地,设计多分辨率视图变换模块,将2D图像特征投影至多分辨率3D特征,并通过融合判别性种子特征实现场景级对齐;引入立方体语义异质性模块,识别每个体素在立方区域内相对于邻近体素的实例级语义重要性;最后设计关键分布对齐模块,以立方体语义异质性为指导,选取关键体素作为实例级锚点,并施加循环损失以确保不同分辨率下关键特征分布的一致性。代码已开源。

原文摘要 · Abstract (English)

Camera-based 3D semantic scene completion (SSC) offers a cost-effective solution for assessing the geometric occupancy and semantic labels of each voxel in the surrounding 3D scene with image inputs, providing a voxel-level scene perception foundation for the perception-prediction-planning autonomous driving systems. Although significant progress has been made in existing methods, their optimization rely solely on the supervision from voxel labels and face the challenge of voxel sparsity as a large portion of voxels in autonomous driving scenarios are empty, which limits both optimization efficiency and model performance. To address this issue, we propose a \textit{Multi-Resolution Alignment (MRA)} approach to mitigate voxel sparsity in camera-based 3D semantic scene completion, which exploits the scene and instance level alignment across multi-resolution 3D features as auxiliary supervision. Specifically, we first propose the Multi-resolution View Transformer module, which projects 2D image features into multi-resolution 3D features and aligns them at the scene level through fusing discriminative seed features. Furthermore, we design the Cubic Semantic Anisotropy module to identify the instance-level semantic significance of each voxel, accounting for the semantic differences of a specific voxel against its neighboring voxels within a cubic area. Finally, we devise a Critical Distribution Alignment module, which selects critical voxels as instance-level anchors with the guidance of cubic semantic anisotropy, and applies a circulated loss for auxiliary supervision on the critical feature distribution consistency across different resolutions. The code is available at https://github.com/PKU-ICST-MIPL/MRA_TIP.

3D感知语义补全多尺度对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。