通过自适应损失调节，提升稀疏输入下的新视角生成质量
Few-shot NeRF by Adaptive Rendering Loss Regularization
- 分阶段渲染监督+自适应权重学习，对齐位置编码与像素损失频率关系
- 在物体级和复杂场景上均达当前最优，早期学全局结构，后期学细节
- 适合需要少样本高质量3D重建的研究者或工业应用
稀疏输入下的新视角合成对神经辐射场（NeRF）提出巨大挑战。近期研究显示，位置编码（PE）的频率正则化可实现良好的少样本NeRF效果。本文揭示了PE的频率正则化与渲染损失之间存在不一致性，这限制了少样本NeRF生成更高质量新视角的能力。为缓解该问题，提出自适应渲染损失正则化方法AR-NeRF：设计两阶段渲染监督机制与自适应渲染损失权重学习策略，以对齐PE与2D像素监督间的频率关系。由此，AR-NeRF可在训练初期更好地学习全局结构，并在后续过程自适应地捕捉局部细节。大量实验表明，该方法在多个数据集（包括物体级与复杂场景）上均达到最先进性能。
原文摘要 · Abstract (English)
Novel view synthesis with sparse inputs poses great challenges to Neural Radiance Field (NeRF). Recent works demonstrate that the frequency regularization of Positional Encoding (PE) can achieve promising results for few-shot NeRF. In this work, we reveal that there exists an inconsistency between the frequency regularization of PE and rendering loss. This prevents few-shot NeRF from synthesizing higher-quality novel views. To mitigate this inconsistency, we propose Adaptive Rendering loss regularization for few-shot NeRF, dubbed AR-NeRF. Specifically, we present a two-phase rendering supervision and an adaptive rendering loss weight learning strategy to align the frequency relationship between PE and 2D-pixel supervision. In this way, AR-NeRF can learn global structures better in the early training phase and adaptively learn local details throughout the training process. Extensive experiments show that our AR-NeRF achieves state-of-the-art performance on different datasets, including object-level and complex scenes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。