用分割引导改进NeRF，更好合成城市街景。
Segmentation-Guided Neural Radiance Fields for Novel Street View Synthesis
- 结合分割图与NeRF，提升对动态物体和天空的建模能力。
- 在街景数据集上，新方法显著减少伪影，细节更清晰。
- 适合做城市级3D重建的研究者或应用开发者。
近期神经辐射场（NeRF）在室内和小规模场景的三维重建与新视角合成方面表现优异。然而，将其扩展到大规模户外环境仍面临动态物体、相机稀疏、纹理缺失及光照变化等挑战。本文提出一种基于分割引导的NeRF增强方法，聚焦复杂城市街景。该方法在ZipNeRF基础上引入基于Grounded SAM的分割掩码生成，有效处理动态物体、建模天空并正则化地面；同时引入外观嵌入以适应不同视角序列间的光照不一致。实验表明，该方法优于基线模型ZipNeRF，显著提升新视角合成质量，减少伪影并增强细节清晰度。
原文摘要 · Abstract (English)
Recent advances in Neural Radiance Fields (NeRF) have shown great potential in 3D reconstruction and novel view synthesis, particularly for indoor and small-scale scenes. However, extending NeRF to large-scale outdoor environments presents challenges such as transient objects, sparse cameras and textures, and varying lighting conditions. In this paper, we propose a segmentation-guided enhancement to NeRF for outdoor street scenes, focusing on complex urban environments. Our approach extends ZipNeRF and utilizes Grounded SAM for segmentation mask generation, enabling effective handling of transient objects, modeling of the sky, and regularization of the ground. We also introduce appearance embeddings to adapt to inconsistent lighting across view sequences. Experimental results demonstrate that our method outperforms the baseline ZipNeRF, improving novel view synthesis quality with fewer artifacts and sharper details.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。