arXiv:2507.01305cs.CVcs.GR2025-07TPAMI被引 10

用单张照片快速生成高质量光照贴图,60倍提速且效果更稳定。

DiffusionLight-Turbo: Accelerated Light Probes for Free via Single-Pass Chrome Ball Inpainting

  • 将Chrome球补全作为扩散模型任务,利用多轮输出取中值生成稳定光照先验。
  • 通过微调曝光LoRA生成多曝光图像并融合,实现高动态范围光照贴图。
  • 引入Turbo LoRA与单次去噪流程,推理时间从30分钟缩短至30秒。

我们提出一种简单高效的方法,仅需一张低动态范围(LDR)图像即可估计光照,将该任务重新定义为Chrome球补全问题。该方法利用预训练的Stable Diffusion XL扩散模型,克服了依赖有限HDR全景数据集的现有方法在泛化能力上的缺陷。尽管概念简单,但扩散模型常生成不一致或错误内容,且难以直接生成HDR格式的Chrome球。分析发现,补全过程对初始噪声高度敏感,常导致不真实输出。为此,我们首先提出DiffusionLight,通过多轮补全计算多个输出的中值,作为稳定的低频光照先验,引导生成高质量最终结果。为生成高动态范围(HDR)光照贴图,我们微调一个Exposure LoRA,生成多曝光的LDR图像并进行融合。虽然有效,DiffusionLight耗时约30分钟。为降低开销,我们引入DiffusionLight-Turbo,将运行时间降至约30秒,实现60倍加速。该速度提升通过训练Turbo LoRA直接预测迭代过程的平均Chrome球实现,并采用LoRA切换技术将推理解耦为单次去噪过程。实验表明,该方法在多种场景下均能生成逼真的光照估计,且在真实世界场景中展现出优异的泛化能力。代码已开源:https://diffusionlight.github.io/turbo

原文摘要 · Abstract (English)

We introduce a simple yet effective technique for estimating lighting from a single low-dynamic-range (LDR) image by reframing the task as a chrome ball inpainting problem. This approach leverages a pre-trained diffusion model, Stable Diffusion XL, to overcome the generalization failures of existing methods that rely on limited HDR panorama datasets. While conceptually simple, the task remains challenging because diffusion models often insert incorrect or inconsistent content and cannot readily generate chrome balls in HDR format. Our analysis reveals that the inpainting process is highly sensitive to the initial noise in the diffusion process, occasionally resulting in unrealistic outputs. To address this, we first introduce DiffusionLight, which uses iterative inpainting to compute a median chrome ball from multiple outputs to serve as a stable, low-frequency lighting prior that guides the generation of a high-quality final result. To generate high-dynamic-range (HDR) light probes, an Exposure LoRA is fine-tuned to create LDR images at multiple exposure values, which are then merged. While effective, DiffusionLight is time-intensive, requiring approximately 30 minutes per estimation. To reduce this overhead, we introduce DiffusionLight-Turbo, which reduces the runtime to about 30 seconds with minimal quality loss. This 60x speedup is achieved by training a Turbo LoRA to directly predict the averaged chrome balls from the iterative process. Inference is further streamlined into a single denoising pass using a LoRA swapping technique. Experimental results that show our method produces convincing light estimates across diverse settings and demonstrates superior generalization to in-the-wild scenarios. Our code is available at https://diffusionlight.github.io/turbo

光照估计扩散模型单图重建实时渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。