arXiv:2509.11171cs.CV2025-09中稿 · ACM MM 2025被引 2

融合语义与物理信息,提升自动驾驶3D场景补全的细节真实感

SPHERE: Semantic-PHysical Engaged REpresentation for 3D Semantic Scene Completion

论文配图:SPHERE: Semantic-PHysical Engaged REpresentation for 3D Semantic Scene Completion
图 1 · 摘自论文原文
  • 用双分支结构引导高斯点初始化,加速训练并聚焦关键区域
  • 引入语义球谐函数建模物理上下文,使几何与语义更一致
  • 在SemanticKITTI等基准上实现领先性能,适合追求真实感的场景重建研究

基于摄像头的3D语义场景补全(SSC)是自动驾驶系统中实现整体场景感知的关键任务,需在体素级别同时还原几何结构与语义信息。现有基于体素和基于平面的方法虽有进展,但难以捕捉真实世界的物理规律以生成精细几何;而NeRF、3DGS等神经重建方法虽具更强物理感知能力,却因计算开销大、收敛慢,在大规模复杂驾驶场景中表现不佳,导致语义精度下降。为此,本文提出语义-物理协同表示框架SPHERE,融合体素与高斯表示,联合利用语义与物理信息。首先,语义引导的高斯初始化(SGI)模块通过双分支3D结构定位关键体素作为锚点,指导高效高斯初始化;其次,物理感知谐波增强(PHE)模块引入语义球谐函数,建模物理相关的上下文细节,并通过焦点分布对齐促进语义-几何一致性,生成具有真实细节的补全结果。在SemanticKITTI与SSCBench-KITTI-360两个主流基准上的大量实验验证了SPHERE的有效性。代码已公开于https://github.com/PKU-ICST-MIPL/SPHERE_ACMMM2025。

原文摘要 · Abstract (English)

Camera-based 3D Semantic Scene Completion (SSC) is a critical task in autonomous driving systems, assessing voxel-level geometry and semantics for holistic scene perception. While existing voxel-based and plane-based SSC methods have achieved considerable progress, they struggle to capture physical regularities for realistic geometric details. On the other hand, neural reconstruction methods like NeRF and 3DGS demonstrate superior physical awareness, but suffer from high computational cost and slow convergence when handling large-scale, complex autonomous driving scenes, leading to inferior semantic accuracy. To address these issues, we propose the Semantic-PHysical Engaged REpresentation (SPHERE) for camera-based SSC, which integrates voxel and Gaussian representations for joint exploitation of semantic and physical information. First, the Semantic-guided Gaussian Initialization (SGI) module leverages dual-branch 3D scene representations to locate focal voxels as anchors to guide efficient Gaussian initialization. Then, the Physical-aware Harmonics Enhancement (PHE) module incorporates semantic spherical harmonics to model physical-aware contextual details and promote semantic-geometry consistency through focal distribution alignment, generating SSC results with realistic details. Extensive experiments and analyses on the popular SemanticKITTI and SSCBench-KITTI-360 benchmarks validate the effectiveness of SPHERE. The code is available at https://github.com/PKU-ICST-MIPL/SPHERE_ACMMM2025.

3D场景补全语义感知高斯表示自动驾驶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。