融合激光雷达与视觉数据，实现高精度三维场景重建与渲染
Neural Surface Reconstruction and Rendering for LiDAR-Visual Systems
- 联合使用NeRF与NDF，从点云和图像中同时恢复外观与结构
- 通过可见性感知占据图解决几何与外观之间的间隙问题
- 适合需要高保真三维重建的自动驾驶与机器人应用
本文提出一种统一的表面重建与渲染框架，用于激光雷达-视觉系统，结合神经辐射场（NeRF）与神经距离场（NDF），从带位姿的图像与点云中恢复场景的外观与结构信息。针对NeRF与NDF之间的结构可见性差距，引入可见性感知占据图，将空间划分为自由、占据、可见未知与背景四类区域，促进场景完整外观与结构的恢复。通过空间可变尺度的SDF-to-density变换，统一训练NDF与NeRF，实现结构与外观的多级细节控制。利用学习到的NDF指导结构感知的NeRF训练，采用自适应球面追踪采样策略实现精准结构渲染；同时，NeRF反向优化补充NDF中缺失或模糊的结构。大量实验表明该方法在多种场景下均具备优异质量与泛化能力。代码将公开于 https://github.com/hku-mars/M2Mapping。
原文摘要 · Abstract (English)
This paper presents a unified surface reconstruction and rendering framework for LiDAR-visual systems, integrating Neural Radiance Fields (NeRF) and Neural Distance Fields (NDF) to recover both appearance and structural information from posed images and point clouds. We address the structural visible gap between NeRF and NDF by utilizing a visible-aware occupancy map to classify space into the free, occupied, visible unknown, and background regions. This classification facilitates the recovery of a complete appearance and structure of the scene. We unify the training of the NDF and NeRF using a spatial-varying scale SDF-to-density transformation for levels of detail for both structure and appearance. The proposed method leverages the learned NDF for structure-aware NeRF training by an adaptive sphere tracing sampling strategy for accurate structure rendering. In return, NeRF further refines structural in recovering missing or fuzzy structures in the NDF. Extensive experiments demonstrate the superior quality and versatility of the proposed method across various scenarios. To benefit the community, the codes will be released at \url{https://github.com/hku-mars/M2Mapping}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。