无需训练即可通过光轨迹控制视频光照变化。
LightCtrl: Training-free Controllable Video Relighting
- 用光轨迹噪声注入实现光照一致性控制。
- 结合几何信息抑制原光照,提升轨迹跟随精度。
- 适合影视后期与虚拟场景光照编辑用户。
近期扩散模型在图像重光照领域取得显著进展,并迅速拓展至视频重光照。然而,现有方法对输出光照的显式控制能力有限。本文提出 LightCtrl,首个无需训练的可控视频重光照方法,可通过用户提供的光轨迹实现显式光照控制。该方法结合预训练扩散模型:先对每帧独立进行图像重光照,再通过视频扩散先验增强时序一致性。为实现动态光照的显式控制,提出两个关键组件:首先,光图注入模块采样与光轨迹相关的噪声并注入源视频潜在表示,提升光照与条件光轨迹的一致性;其次,几何感知重光照模块在频域动态融合RGB与法向图潜在表示,抑制原始光照影响,进一步强化对输入光轨迹的遵循。实验表明,LightCtrl能生成高质量视频,光照变化多样且紧密跟随指定光轨迹,相比基线方法显著提升可控性。代码已开源:https://github.com/GVCLab/LightCtrl。
原文摘要 · Abstract (English)
Recent diffusion models have achieved remarkable success in image relighting, and this success has quickly been extended to video relighting. However, existing methods offer limited explicit control over illumination in the relighted output. We present LightCtrl, the first controllable video relighting method that enables explicit control of video illumination through a user-supplied light trajectory in a training-free manner. Our approach combines pre-trained diffusion models: an image relighting model processes each frame individually, followed by a video diffusion prior to enhance temporal consistency. To achieve explicit control over dynamically varying lighting, we introduce two key components. First, a Light Map Injection module samples light trajectory-specific noise and injects it into the latent representation of the source video, improving illumination coherence with the conditional light trajectory. Second, a Geometry-Aware Relighting module dynamically combines RGB and normal map latents in the frequency domain to suppress the influence of the original lighting, further enhancing adherence to the input light trajectory. Experiments show that LightCtrl produces high-quality videos with diverse illumination changes that closely follow the specified light trajectory, demonstrating improved controllability over baseline methods. Code is available at: https://github.com/GVCLab/LightCtrl.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。