用扩散模型实现图像光影的精细参数化控制
LightLab: Controlling Light Sources in Images with Diffusion Models
- 基于真实与合成图像对微调扩散模型，实现光影可控生成
- 支持对光源强度和颜色的精确调节，效果优于现有方法
- 适合需要精准光影编辑的数字内容创作人群
我们提出一种简单而有效的基于扩散模型的方法，实现图像中光源的细粒度、参数化控制。现有重光照方法要么依赖多视角输入在推理时进行逆渲染，要么无法提供对光照变化的显式控制。本方法在少量真实原始照片对的基础上，结合大规模合成渲染图像，微调扩散模型以激发其逼真的光照先验。利用光照的线性特性，合成展示目标光源或环境光变化的图像对。通过该数据集与合适的微调策略，训练出可精确调整光照且具有显式控制能力的模型。最后，我们展示了该方法在光照编辑上的出色效果，并在用户偏好测试中优于现有方法。
原文摘要 · Abstract (English)
We present a simple, yet effective diffusion-based method for fine-grained, parametric control over light sources in an image. Existing relighting methods either rely on multiple input views to perform inverse rendering at inference time, or fail to provide explicit control over light changes. Our method fine-tunes a diffusion model on a small set of real raw photograph pairs, supplemented by synthetically rendered images at scale, to elicit its photorealistic prior for relighting. We leverage the linearity of light to synthesize image pairs depicting controlled light changes of either a target light source or ambient illumination. Using this data and an appropriate fine-tuning scheme, we train a model for precise illumination changes with explicit control over light intensity and color. Lastly, we show how our method can achieve compelling light editing results, and outperforms existing methods based on user preference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。