用4D时空建模解决动态场景高动态范围成像的鬼影问题。
HDR-NSFF: High Dynamic Range Neural Scene Flow Fields
- 从2D对齐转向4D连续场建模,统一处理光照、运动与几何。
- 在真实动态场景下实现精细辐射率恢复与连贯动态表现。
- 适合做高动态范围视频生成与3D场景重建的研究者。
真实场景的辐射亮度通常远超标准相机捕捉范围。传统HDR方法依赖多曝光图像拼接,但受限于2D像素级对齐,在动态场景中易产生鬼影和时间不一致问题。为此,我们提出HDR-NSFF,将方法范式从2D融合转向4D时空建模。该框架通过空间时间连续函数表示动态HDR辐射场,兼容神经辐射场与4D高斯点云(4DGS)等动态表征。整个端到端流程显式建模HDR辐射、3D场景光流、几何结构与色调映射,保证物理合理性与全局一致性。为提升鲁棒性,我们(1)引入DINO特征扩展语义光流,实现曝光不变的运动估计;(2)加入生成先验作为正则项,补偿单目观测不足及过曝信息损失。我们构建了首个专为动态HDR场景设计的真实世界数据集HDR-GoPro。实验表明,即使在剧烈曝光变化下,HDR-NSFF仍能恢复精细辐射细节并保持动态连贯性,显著优于现有方法。
原文摘要 · Abstract (English)
Radiance of real-world scenes typically spans a much wider dynamic range than what standard cameras can capture. While conventional HDR methods merge alternating-exposure frames, these approaches are inherently constrained to 2D pixel-level alignment, often leading to ghosting artifacts and temporal inconsistency in dynamic scenes. To address these limitations, we present HDR-NSFF, a paradigm shift from 2D-based merging to 4D spatio-temporal modeling. Our framework reconstructs dynamic HDR radiance fields from alternating-exposure monocular videos by representing the scene as a continuous function of space and time, and is compatible with both neural radiance field and 4D Gaussian Splatting (4DGS) based dynamic representations. This unified end-to-end pipeline explicitly models HDR radiance, 3D scene flow, geometry, and tone-mapping, ensuring physical plausibility and global coherence. We further enhance robustness by (i) extending semantic-based optical flow with DINO features to achieve exposure-invariant motion estimation, and (ii) incorporating a generative prior as a regularizer to compensate for limited observation in monocular captures and saturation-induced information loss. To evaluate HDR space-time view synthesis, we present the first real-world HDR-GoPro dataset specifically designed for dynamic HDR scenes. Experiments demonstrate that HDR-NSFF recovers fine radiance details and coherent dynamics even under challenging exposure variations, thereby achieving state-of-the-art performance in novel space-time view synthesis. Project page: https://shin-dong-yeon.github.io/HDR-NSFF/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。