arXiv:2509.23769cs.GRcs.AI2025-09被引 1

将图像光照重置技术扩展到视频,实现动态光影自由控制。

ReLumix: Extending Image Relighting to Video via Video Diffusion Models

  • 分两阶段处理:先选参考帧重光,再用SVD模型传播光照
  • 在真实视频上仍保持高视觉质量,支持任意图像重光方法
  • 适合影视后期、虚拟拍摄等需要灵活控光的场景

视频后期的光照控制是计算摄影中的关键挑战。现有方法灵活性不足,仅支持特定重光模型。本文提出ReLumix,将重光算法与时间合成解耦,使任意图像重光技术(如扩散模型、物理渲染器)可无缝应用于视频。该框架采用两阶段流程:(1) 艺术家使用任一图像重光方法对单个参考帧进行光照调整;(2) 通过微调的稳定视频扩散(SVD)模型将目标光照沿时间序列平滑传播。为保证时序一致性并减少伪影,引入门控交叉注意力机制实现特征平滑融合,并采用基于SVD运动先验的时间自举策略。尽管在合成数据上训练,ReLumix在真实视频上仍表现出良好泛化能力,显著提升视觉保真度,提供了一种可扩展、通用的动态光照控制方案。

原文摘要 · Abstract (English)

Controlling illumination during video post-production is a crucial yet elusive goal in computational photography. Existing methods often lack flexibility, restricting users to certain relighting models. This paper introduces ReLumix, a novel framework that decouples the relighting algorithm from temporal synthesis, thereby enabling any image relighting technique to be seamlessly applied to video. Our approach reformulates video relighting into a simple yet effective two-stage process: (1) an artist relights a single reference frame using any preferred image-based technique (e.g., Diffusion Models, physics-based renderers); and (2) a fine-tuned stable video diffusion (SVD) model seamlessly propagates this target illumination throughout the sequence. To ensure temporal coherence and prevent artifacts, we introduce a gated cross-attention mechanism for smooth feature blending and a temporal bootstrapping strategy that harnesses SVD's powerful motion priors. Although trained on synthetic data, ReLumix shows competitive generalization to real-world videos. The method demonstrates significant improvements in visual fidelity, offering a scalable and versatile solution for dynamic lighting control.

视频生成扩散模型光照重置

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。