混合方法提升视频光影重渲染稳定性，适合影视制作流程
Yesnt: Are Diffusion Relighting Models Ready for Capture Stage Compositing? A Hybrid Alternative to Bridge the Gap
- 用扩散模型提取材质先验，结合光流引导实现时序稳定
- 在真实与合成数据上显著优于纯扩散基线，支持更长序列
- 适合影视级捕获视频的后期合成，兼顾生成与物理规律
体积视频重光照对将实拍表演融入虚拟世界至关重要，但现有方法难以实现时序稳定且可投入生产的输出。基于扩散的内在分解方法在单帧上表现良好，但扩展到序列时面临随机噪声和不稳定性问题，而视频扩散模型受限于内存与规模。本文提出一种混合重光照框架，融合扩散生成的材质先验、时序正则化与物理驱动渲染。通过光流引导，聚合多帧材质属性的随机估计，生成时序一致的阴影成分；对阴影、反射等间接光照，则从高斯透明场提取网格代理，并在标准图形管线中渲染。在真实与合成数据上的实验表明，该方法相比纯扩散基线显著提升序列稳定性，且可处理超过视频扩散模型可行长度的片段。结果表明，融合学习先验与物理约束的混合策略是迈向生产级体积视频重光照的可行路径。
原文摘要 · Abstract (English)
Volumetric video relighting is essential for bringing captured performances into virtual worlds, but current approaches struggle to deliver temporally stable, production-ready results. Diffusion-based intrinsic decomposition methods show promise for single frames, yet suffer from stochastic noise and instability when extended to sequences, while video diffusion models remain constrained by memory and scale. We propose a hybrid relighting framework that combines diffusion-derived material priors with temporal regularization and physically motivated rendering. Our method aggregates multiple stochastic estimates of per-frame material properties into temporally consistent shading components, using optical-flow-guided regularization. For indirect effects such as shadows and reflections, we extract a mesh proxy from Gaussian Opacity Fields and render it within a standard graphics pipeline. Experiments on real and synthetic captures show that this hybrid strategy achieves substantially more stable relighting across sequences than diffusion-only baselines, while scaling beyond the clip lengths feasible for video diffusion. These results indicate that hybrid approaches, which balance learned priors with physically grounded constraints, are a practical step toward production-ready volumetric video relighting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。