用视图一致性分布替代固定深度值，提升NeRF在真实场景下的重建质量。
A View-consistent Sampling Method for Regularized Training of Neural Radiance Fields
- 用2D像素点的低层颜色与高层特征构建视图一致性分布
- 在多个公开数据集上显著优于现有NeRF方法和深度正则化技术
- 适合追求高质量3D重建的视觉算法研究者
神经辐射场（NeRF）已成为场景表示与三维重建的重要框架。为提升其在真实世界数据上的表现，深度正则化被证明最有效。然而，深度估计模型不仅需要昂贵的3D监督，还存在泛化问题，尤其在户外无界场景中容易出错。本文提出用视图一致性分布替代固定深度值来正则化NeRF训练。具体而言，通过利用从每条射线采样3D点投影到2D像素位置的低层颜色特征和高层蒸馏特征，计算视图一致性分布。通过从该分布中采样，对NeRF训练施加隐式正则化。同时结合深度推挤损失，协同提供有效正则化以消除失败模式。在多个公共数据集上的大量实验表明，所提方法在新视角合成效果上显著优于当前最先进的NeRF变体及不同深度正则化方法。
原文摘要 · Abstract (English)
Neural Radiance Fields (NeRF) has emerged as a compelling framework for scene representation and 3D recovery. To improve its performance on real-world data, depth regularizations have proven to be the most effective ones. However, depth estimation models not only require expensive 3D supervision in training, but also suffer from generalization issues. As a result, the depth estimations can be erroneous in practice, especially for outdoor unbounded scenes. In this paper, we propose to employ view-consistent distributions instead of fixed depth value estimations to regularize NeRF training. Specifically, the distribution is computed by utilizing both low-level color features and high-level distilled features from foundation models at the projected 2D pixel-locations from per-ray sampled 3D points. By sampling from the view-consistency distributions, an implicit regularization is imposed on the training of NeRF. We also utilize a depth-pushing loss that works in conjunction with the sampling technique to jointly provide effective regularizations for eliminating the failure modes. Extensive experiments conducted on various scenes from public datasets demonstrate that our proposed method can generate significantly better novel view synthesis results than state-of-the-art NeRF variants as well as different depth regularization methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。