无需训练即可实现极端视角下的4D视频光照重置。
Light4D: Training-Free Extreme Viewpoint 4D Video Relighting
- 通过解耦光流引导，在潜空间注入光照控制并保持几何一致。
- 在-90到90度相机旋转下仍保持时间一致性，无明显闪烁。
- 适合需要快速部署、无需数据训练的光照重制场景。
基于扩散生成模型的图像与视频光照重置已取得进展，但扩展至4D光照重置仍面临配对4D光照数据稀缺和极端视角下时序一致性难维持的挑战。本文提出Light4D，一种无需训练的框架，可在目标光照下合成一致的4D视频，即使在极端视角变化下也表现稳健。首先，引入解耦光流引导（Disentangled Flow Guidance），一种时间感知策略，有效将光照控制注入潜空间，同时保持几何完整性。其次，为强化时序一致性，在IC-Light架构中设计时序一致注意力，并加入确定性正则化以消除外观闪烁。大量实验表明，该方法在时序一致性和光照保真度上表现优异，可稳定处理-90°至90°的相机旋转。代码与网页见：https://github.com/AIGeeksGroup/Light4D, https://aigeeksgroup.github.io/Light4D。
原文摘要 · Abstract (English)
Recent advances in diffusion-based generative models have established a new paradigm for image and video relighting. However, extending these capabilities to 4D relighting remains challenging, due primarily to the scarcity of paired 4D relighting training data and the difficulty of maintaining temporal consistency across extreme viewpoints. In this work, we propose Light4D, a novel training-free framework designed to synthesize consistent 4D videos under target illumination, even under extreme viewpoint changes. First, we introduce Disentangled Flow Guidance, a time-aware strategy that effectively injects lighting control into the latent space while preserving geometric integrity. Second, to reinforce temporal consistency, we develop Temporal Consistent Attention within the IC-Light architecture and further incorporate deterministic regularization to eliminate appearance flickering. Extensive experiments demonstrate that our method achieves competitive performance in temporal consistency and lighting fidelity, robustly handling camera rotations from -90 to 90. Code: https://github.com/AIGeeksGroup/Light4D. Website: https://aigeeksgroup.github.io/Light4D.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。