自监督分解动态场景，重建高保真城市驾驶表面
DeSiRe-GS: 4D Street Gaussians for Static-Dynamic Decomposition and Surface Reconstruction for Urban Driving Scenes
- 利用2D运动掩码引导3D高斯点云，实现动静分离
- 通过几何正则化避免稀疏数据过拟合，还原真实物体表面
- 无需外部标注，适合自动驾驶场景的实时重建
我们提出DeSiRe-GS，一种自监督的高斯点阵表示方法，可在复杂驾驶场景中有效实现静态-动态分解并完成高保真表面重建。该方法采用两阶段优化流程：第一阶段基于3D高斯点阵仅能重建静态区域的特性，提取2D运动掩码；第二阶段将这些运动先验以可微方式映射至高斯空间，结合引入的几何正则化，解决了自动驾驶中数据稀疏导致的过拟合问题，重建出与物体表面一致而非漂浮于空中的物理合理高斯点云。此外，通过时间跨视角一致性约束，确保时空一致性，实现高质量表面重建。大量实验表明，DeSiRe-GS在效率与效果上均优于现有自监督方法，精度接近依赖外部3D边界框标注的方法。代码已开源。
原文摘要 · Abstract (English)
We present DeSiRe-GS, a self-supervised gaussian splatting representation, enabling effective static-dynamic decomposition and high-fidelity surface reconstruction in complex driving scenarios. Our approach employs a two-stage optimization pipeline of dynamic street Gaussians. In the first stage, we extract 2D motion masks based on the observation that 3D Gaussian Splatting inherently can reconstruct only the static regions in dynamic environments. These extracted 2D motion priors are then mapped into the Gaussian space in a differentiable manner, leveraging an efficient formulation of dynamic Gaussians in the second stage. Combined with the introduced geometric regularizations, our method are able to address the over-fitting issues caused by data sparsity in autonomous driving, reconstructing physically plausible Gaussians that align with object surfaces rather than floating in air. Furthermore, we introduce temporal cross-view consistency to ensure coherence across time and viewpoints, resulting in high-quality surface reconstruction. Comprehensive experiments demonstrate the efficiency and effectiveness of DeSiRe-GS, surpassing prior self-supervised arts and achieving accuracy comparable to methods relying on external 3D bounding box annotations. Code is available at https://github.com/chengweialan/DeSiRe-GS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。