用生成模型实现高保真人体自由视角渲染与环境光重光照。
Generative Relightable Avatars
- 结合物理规律与生成式优化，分步提升渲染细节。
- 在真实环境光下实现逼真、时序连贯的全身动态渲染。
- 适合影视特效与虚拟人开发人员使用。
我们提出生成式可重光照化身（GRA），一种针对个体的全身人体照片级自由视角渲染与环境图重光照方法。我们认为建模精细外观细节本质上是多对一问题，适合采用生成式框架。与完全回归式的重光照化身方法不同，GRA 采用混合策略，结合可控的物理基础重光照与概率性细化。从追踪到的动画网格出发，我们在纹理空间优化材质参数，并在目标HDR环境图下渲染粗略重光照结果。随后，通过前馈模型细化纹理，捕捉姿态相关的纹理动态和超出简化反射假设的光照效果。最后，经微调的视频到视频扩散模型将物理基础渲染转化为时间连贯、高细节的视频，同时保持3D控制，采用误差回收策略生成长视频。实验表明，该方法在感知质量上优于现有重光照化身基线。
原文摘要 · Abstract (English)
We present Generative Relightable Avatars (GRA), a person-specific method for photorealistic free-view rendering and environment-map relighting of full-body humans. We postulate that modeling fine-grained appearance details is inherently a one-to-many problem that can benefit from a generative formulation. In contrast to fully regressive relightable avatar methods, GRA follows a hybrid approach that combines controllable, physics-grounded relighting with probabilistic refinement. Starting from a tracked animated mesh, we optimize material parameters in UV-space and render a coarse relit appearance under a target HDR environment map. Next, we refine the textures with a feed-forward model to capture pose-dependent texture dynamics and illumination effects beyond simplified reflectance assumptions. Finally, a fine-tuned video-to-video diffusion model transforms the physically grounded renderings into temporally coherent, high-detail videos while preserving 3D control, with an error-recycling strategy for generating long videos. Experimental evaluations demonstrate our method's improved perceptual quality over prior relightable avatar baselines. Project Page: https://vcai.mpi-inf.mpg.de/projects/GRA/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。