用高斯渲染提升3D占据预测的几何一致性,消除漂浮伪影。
GaussRender: Learning 3D Occupancy with Gaussian Rendering
- 通过高斯点云可微渲染,强制预测与真实占据在2D视图中投影一致。
- 在nuScenes和KITTI360数据集上,表面敏感指标RayIoU显著提升。
- 适合作为通用模块嵌入现有3D占据模型,无需推理修改。
理解驾驶场景的3D几何与语义对自动驾驶安全至关重要。近期3D占据预测虽提升了场景表示,但常因视觉不一致导致漂浮伪影和表面定位不准。传统体素级损失(如交叉熵)无法保证可见几何一致性。本文提出GaussRender,通过投影预测与真实占据到2D相机视图并施加监督,惩罚产生不一致2D投影的3D配置,从而强化3D结构一致性。采用可微渲染与高斯点阵实现高效计算。GaussRender可无缝集成至现有架构,无需推理时修改。在SurroundOcc-nuScenes、Occ3D-nuScenes、SSCBench-KITTI360等多个基准上评估,显著提升多种3D占据模型(TPVFormer、SurroundOcc、Symphonies)的几何保真度,尤其在表面敏感指标RayIoU上达到当前最优。代码已开源:https://github.com/valeoai/GaussRender。
原文摘要 · Abstract (English)
Understanding the 3D geometry and semantics of driving scenes is critical for safe autonomous driving. Recent advances in 3D occupancy prediction have improved scene representation but often suffer from visual inconsistencies, leading to floating artifacts and poor surface localization. Existing voxel-wise losses (e.g., cross-entropy) fail to enforce visible geometric coherence. In this paper, we propose GaussRender, a module that improves 3D occupancy learning by enforcing projective consistency. Our key idea is to project both predicted and ground-truth 3D occupancy into 2D camera views, where we apply supervision. Our method penalizes 3D configurations that produce inconsistent 2D projections, thereby enforcing a more coherent 3D structure. To achieve this efficiently, we leverage differentiable rendering with Gaussian splatting. GaussRender seamlessly integrates with existing architectures while maintaining efficiency and requiring no inference-time modifications. Extensive evaluations on multiple benchmarks (SurroundOcc-nuScenes, Occ3D-nuScenes, SSCBench-KITTI360) demonstrate that GaussRender significantly improves geometric fidelity across various 3D occupancy models (TPVFormer, SurroundOcc, Symphonies), achieving state-of-the-art results, particularly on surface-sensitive metrics such as RayIoU. The code is open-sourced at https://github.com/valeoai/GaussRender.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。