arXiv:2605.21766cs.CVcs.GR2026-05

用扩散模型实现人体视频光影的稳定重光照,效果逼真且动态可控。

BodyReLux: Temporally Consistent Full-Body Video Relighting

论文配图:BodyReLux: Temporally Consistent Full-Body Video Relighting
图 1 · 摘自论文原文
  • 基于视频扩散模型,结合动态光照捕获技术训练
  • 支持主体特定的高质量、时序一致重光照,效果逼真
  • 适合影视后期与虚拟内容创作,尤其需精准光影控制场景

人体表演的重光照是后期制作与内容创作中的基础任务。我们提出 BodyReLux,一种基于扩散模型的主体特定视频重光照框架,可实现全身人体表演在时间上一致的重光照。模型在混合数据集上训练,包含像素对齐的视频重光照配对,涵盖多样化的光照条件、表演动作和视角。为构建该数据集,我们结合传统的静态逐光采集(One-Light-at-a-Time, OLAT)与一种新型动态表演采集方法:将两条平滑变化的光照序列快速交替呈现。由于光照频率高于人眼闪烁融合阈值,交替不会产生闪烁感。我们从预训练的文本到视频模型出发进行训练,以充分利用生成先验。为实现精确光照控制,引入一种新光照条件方法,将每个光源表示为一个令牌,并使用掩码注意力机制对光照序列进行建模,支持动态光照调节。配合精心设计的数据增强流程,实现了主体特定人体表演的逼真、鲁棒且时序一致的视频重光照。

原文摘要 · Abstract (English)

Being able to relight human performance is a fundamental task for post production and content creation. We present BodyReLux, a subject-specific video diffusion-based framework for relighting full-body human performances in a temporally consistent way. Our model is trained on a hybrid dataset of pixel-aligned video relighting pairs, covering a diverse combination of lighting conditions, performances and viewpoints. To acquire such dataset, we combine traditional static One-Light-at-a-Time (OLAT) capture and a novel dynamic performance capture in which two smoothly varying lighting sequences are rapidly interleaved. Because the lighting operates above the human flicker-fusion threshold, the interleaving does not appear to strobe. We train our video relighting model from a pretrained text-to-video model to fully leverage the generative priors for producing high quality videos. To achieve accurate lighting control, we introduce a new lighting conditioning method that represents each light source as a token. We further condition on sequences of lighting using masked attention to support dynamic lighting control. Together with a carefully designed data augmentation pipeline, we achieve photorealistic, robust, and temporally consistent video relighting of subject-specific human performances.

视频重光照扩散模型时序一致人体重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。