arXiv:2502.08590cs.CV2025-02ICCV被引 42

无需训练即可实现稳定视频光影重演,解决闪烁问题。

Light-A-Video: Training-free Video Relighting via Progressive Light Fusion

  • 用跨帧注意力增强背景光照一致性
  • 通过渐进光融合实现光照平滑过渡
  • 适合需要快速部署的视频光影编辑场景

近期基于大规模数据集和预训练扩散模型的图像光影重演技术已能实现一致光照。但视频光影重演仍落后,主要因训练成本高及高质量视频数据集稀缺。逐帧应用图像重演模型会导致光照源不一致和外观不一致,引发视频闪烁。本文提出 Light-A-Video,一种无需训练的时序平稳视频光影重演方法。在图像重演模型基础上引入两项关键技术:一是设计一致光照注意力(CLA)模块,增强自注意力层中的跨帧交互,稳定背景光照生成;二是基于光照传输独立性物理原理,采用线性混合策略,结合源视频与重演外观,通过渐进光融合(PLF)确保光照变化的时序平滑。实验表明,Light-A-Video在保持图像重演质量的同时,显著提升视频时序一致性,实现帧间光照过渡连贯。项目主页:https://bujiazi.github.io/light-a-video.github.io/

原文摘要 · Abstract (English)

Recent advancements in image relighting models, driven by large-scale datasets and pre-trained diffusion models, have enabled the imposition of consistent lighting. However, video relighting still lags, primarily due to the excessive training costs and the scarcity of diverse, high-quality video relighting datasets. A simple application of image relighting models on a frame-by-frame basis leads to several issues: lighting source inconsistency and relighted appearance inconsistency, resulting in flickers in the generated videos. In this work, we propose Light-A-Video, a training-free approach to achieve temporally smooth video relighting. Adapted from image relighting models, Light-A-Video introduces two key techniques to enhance lighting consistency. First, we design a Consistent Light Attention (CLA) module, which enhances cross-frame interactions within the self-attention layers of the image relight model to stabilize the generation of the background lighting source. Second, leveraging the physical principle of light transport independence, we apply linear blending between the source video's appearance and the relighted appearance, using a Progressive Light Fusion (PLF) strategy to ensure smooth temporal transitions in illumination. Experiments show that Light-A-Video improves the temporal consistency of relighted video while maintaining the relighted image quality, ensuring coherent lighting transitions across frames. Project page: https://bujiazi.github.io/light-a-video.github.io/.

视频生成光影重演扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。