改进NeRF的频率滤波机制,加速三维场景与相机位姿联合优化。
FA-BARF: Frequency Adapted Bundle-Adjusting Neural Radiance Fields
- 用空间低通滤波替代时间低通滤波,缓解优化过程中的频率波动问题。
- 在物体中心场景中实现小扰动下优化速度提升,支持未知相机位姿重建。
- 适合需要实时3D建模与高精度重建的应用,如自动驾驶、AR/VR。
神经辐射场(NeRF)在逼真新视角合成方面表现优异,但其关键限制在于依赖人工设计的频率退火策略来恢复相机位姿不准确的三维场景。该策略使用时间低通滤波保证收敛,却减缓了隐式场景重建与相机位姿联合优化的进程。本文提出频率自适应捆绑调整神经辐射场(FA-BARF),以频率自适应的空间低通滤波替代时间低通滤波,解决这一瓶颈。我们建立了理论框架,揭示了NeRF位置编码与相机位姿之间的关系,并证明所提滤波能有效缓解时间滤波引发的频率波动。此外,实验表明在多种视角的径向不确定性重叠下,空间低通滤波可有效优化相机位姿。大量实验显示,FA-BARF在物体中心场景中对小扰动具有鲁棒性,显著加速联合优化过程,且能重建真实世界中相机位姿未知的场景。这为实时需求下的密集3D映射与重建提供了更广阔可能性。代码将在论文接收后发布。
原文摘要 · Abstract (English)
Neural Radiance Fields (NeRF) have exhibited highly effective performance for photorealistic novel view synthesis recently. However, the key limitation it meets is the reliance on a hand-crafted frequency annealing strategy to recover 3D scenes with imperfect camera poses. The strategy exploits a temporal low-pass filter to guarantee convergence while decelerating the joint optimization of implicit scene reconstruction and camera registration. In this work, we introduce the Frequency Adapted Bundle Adjusting Radiance Field (FA-BARF), substituting the temporal low-pass filter for a frequency-adapted spatial low-pass filter to address the decelerating problem. We establish a theoretical framework to interpret the relationship between position encoding of NeRF and camera registration and show that our frequency-adapted filter can mitigate frequency fluctuation caused by the temporal filter. Furthermore, we show that applying a spatial low-pass filter in NeRF can optimize camera poses productively through radial uncertainty overlaps among various views. Extensive experiments show that FA-BARF can accelerate the joint optimization process under little perturbations in object-centric scenes and recover real-world scenes with unknown camera poses. This implies wider possibilities for NeRF applied in dense 3D mapping and reconstruction under real-time requirements. The code will be released upon paper acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。