提出神经曝光场,让3D场景重建自动适应光照变化。
Learning Neural Exposure Fields for View Synthesis
- 在3D空间学习每点最优曝光值,联合优化场景与曝光
- 真实场景下比基线提升超55%,训练更快
- 适合室内户外混合光照的复杂拍摄数据
近期神经场景表示技术在3D重建和视角合成上取得突破性进展。然而,对于存在显著曝光差异的真实场景(如室内外混合区域或带窗户的房间),现有方法性能明显下降。本文提出神经曝光场(NExF),一种从挑战性真实采集数据中高保真、3D一致地重建3D场景的新方法。核心思想是学习一个神经场,为每个3D点预测最优曝光值,从而在3D空间中联合优化场景表示与曝光场。不同于相机逐图像/像素选择曝光,我们将其推广至3D空间优化,实现高动态范围场景下的精准视角合成,无需后期处理或多曝光采集。贡献包括:新颖的曝光预测神经表示、基于新神经条件机制的联合优化系统,以及在多个挑战性真实数据集上的卓越表现。实验表明,本方法训练速度优于先前工作,在多个基准上超越最佳基线超过55%。
原文摘要 · Abstract (English)
Recent advances in neural scene representations have led to unprecedented quality in 3D reconstruction and view synthesis. Despite achieving high-quality results for common benchmarks with curated data, outputs often degrade for data that contain per image variations such as strong exposure changes, present, e.g., in most scenes with indoor and outdoor areas or rooms with windows. In this paper, we introduce Neural Exposure Fields (NExF), a novel technique for robustly reconstructing 3D scenes with high quality and 3D-consistent appearance from challenging real-world captures. In the core, we propose to learn a neural field predicting an optimal exposure value per 3D point, enabling us to optimize exposure along with the neural scene representation. While capture devices such as cameras select optimal exposure per image/pixel, we generalize this concept and perform optimization in 3D instead. This enables accurate view synthesis in high dynamic range scenarios, bypassing the need of post-processing steps or multi-exposure captures. Our contributions include a novel neural representation for exposure prediction, a system for joint optimization of the scene representation and the exposure field via a novel neural conditioning mechanism, and demonstrated superior performance on challenging real-world data. We find that our approach trains faster than prior works and produces state-of-the-art results on several benchmarks improving by over 55% over best-performing baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。