无需激光雷达,用深度图和扩散模型重建城市场景几何。
D$^2$GS: Dense Depth Regularization for LiDAR-free Urban Scene Reconstruction
- 用多视角深度预测反投影生成稠密点云,逐步剪枝提升全局一致性。
- 融合扩散模型先验增强深度图,反向优化高斯点云几何结构。
- 约束道路区域的高斯点法向与形状,提升地面精度,适合自动驾驶场景。
最近,高斯点阵(Gaussian Splatting, GS)在自动驾驶领域的城市场景重建中展现出巨大潜力。然而,现有方法通常依赖多模态传感器输入,如激光雷达(LiDAR)和图像。尽管激光雷达提供的几何先验能显著缓解重建中的病态问题,但实际获取精确的激光雷达数据仍具挑战:一是需精确的时空标定,因传感器可能非同步采集;二是当激光雷达与相机安装位置不一致时,会引入重投影误差。为避免获取精确激光雷达深度的困难,本文提出 D²GS,一种无需激光雷达的城市场景重建框架。首先,通过反投影多视角度量深度预测生成稠密点云,并采用渐进式剪枝策略优化其全局一致性。其次,联合优化高斯几何与预测的稠密度量深度,利用深度基础模型的扩散先验增强渲染深度图,进而为高斯训练提供更强几何约束。最后,通过约束道路区域高斯点的形状与法向属性,提升地面几何精度。在 Waymo 数据集上的大量实验表明,本方法持续优于当前最优方法,重建几何精度甚至超过使用真实激光雷达数据的方法。
原文摘要 · Abstract (English)
Recently, Gaussian Splatting (GS) has shown great potential for urban scene reconstruction in the field of autonomous driving. However, current urban scene reconstruction methods often depend on multimodal sensors as inputs, \textit{i.e.} LiDAR and images. Though the geometry prior provided by LiDAR point clouds can largely mitigate ill-posedness in reconstruction, acquiring such accurate LiDAR data is still challenging in practice: i) precise spatiotemporal calibration between LiDAR and other sensors is required, as they may not capture data simultaneously; ii) reprojection errors arise from spatial misalignment when LiDAR and cameras are mounted at different locations. To avoid the difficulty of acquiring accurate LiDAR depth, we propose D$^2$GS, a LiDAR-free urban scene reconstruction framework. In this work, we obtain geometry priors that are as effective as LiDAR while being denser and more accurate. $\textbf{First}$, we initialize a dense point cloud by back-projecting multi-view metric depth predictions. This point cloud is then optimized by a Progressive Pruning strategy to improve the global consistency. $\textbf{Second}$, we jointly refine Gaussian geometry and predicted dense metric depth via a Depth Enhancer. Specifically, we leverage diffusion priors from a depth foundation model to enhance the depth maps rendered by Gaussians. In turn, the enhanced depths provide stronger geometric constraints during Gaussian training. $\textbf{Finally}$, we improve the accuracy of ground geometry by constraining the shape and normal attributes of Gaussians within road regions. Extensive experiments on the Waymo dataset demonstrate that our method consistently outperforms state-of-the-art methods, producing more accurate geometry even when compared with those using ground-truth LiDAR data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。