用扩散模型预测动态场景的高动态光照，生成逼真高清光场图。
Lighting in Motion: Spatiotemporal HDR Lighting Estimation
- 基于扩散模型，通过多曝光镜面与漫射球生成光照先验。
- 在室内室外场景上实现高精度空间光照估计，优于现有方法。
- 适合影视特效、虚拟拍摄等需要真实光照重建的应用。
我们提出 Lighting in Motion（LiMo），一种基于扩散模型的时空光照估计方法，旨在同时实现高频率细节的真实预测与精确照度估计。为此，我们根据输入中3D位置生成一系列不同曝光的镜面与漫射球体；利用扩散先验，在大规模定制化的室内外场景数据集上微调现有强大扩散模型，该数据集包含配对的时空光照探针。为实现精准的空间条件控制，我们证明仅依赖深度信息不足，并引入新几何条件以提供场景相对于目标3D位置的相对位置。最后，通过可微渲染将不同曝光下的漫射与镜面预测融合成单一HDRI地图。我们全面评估了方法及设计选择，确立了LiMo在空间控制与预测精度上的最新技术水平。
原文摘要 · Abstract (English)
We present Lighting in Motion (LiMo), a diffusion-based approach to spatiotemporal lighting estimation. LiMo targets both realistic high-frequency detail prediction and accurate illuminance estimation. To account for both, we propose generating a set of mirrored and diffuse spheres at different exposures, based on their 3D positions in the input. Making use of diffusion priors, we fine-tune powerful existing diffusion models on a large-scale customized dataset of indoor and outdoor scenes, paired with spatiotemporal light probes. For accurate spatial conditioning, we demonstrate that depth alone is insufficient and we introduce a new geometric condition to provide the relative position of the scene to the target 3D position. Finally, we combine diffuse and mirror predictions at different exposures into a single HDRI map leveraging differentiable rendering. We thoroughly evaluate our method and design choices to establish LiMo as state-of-the-art for both spatial control and prediction accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。