实现可重光照的全身高精度虚拟人,支持手脸细节与姿态变化下的真实光影效果。
Relightable Full-Body Gaussian Codec Avatars
- 分拆光照传输为局部与非局部效应,用可旋转的球谐函数建模局部光照。
- 引入阴影网络预测肢体间遮挡,提升复杂姿态下的光影一致性。
- 适用于虚拟试衣、数字人直播等需真实光照交互的场景。
我们提出一种可重光照的全身高斯编码虚拟人建模方法,能够精细还原面部与手部等细节。全身体态变化带来的大形变导致光照传输显著改变,包括局部光照方向变化和肢体间的非局部遮挡。为此,我们将光照传输分解为局部与非局部效应:局部变化采用可学习的区域球谐函数建模漫反射辐射传输,相比球谐函数更高效地适应体姿变换;非局部变化通过阴影网络预测,基于预计算的基础网格入射辐射生成遮挡效果;最后采用延迟着色方法建模镜面反射传输,精准捕捉高光与眼闪光等细节。实验表明,该方法在新光照条件和未见过的姿态下均具备优异泛化能力。
原文摘要 · Abstract (English)
We propose Relightable Full-Body Gaussian Codec Avatars, a new approach for modeling relightable full-body avatars with fine-grained details including face and hands. The unique challenge for relighting full-body avatars lies in the large deformations caused by body articulation and the resulting impact on appearance caused by light transport. Changes in body pose can dramatically change the orientation of body surfaces with respect to lights, resulting in both local appearance changes due to changes in local light transport functions, as well as non-local changes due to occlusion between body parts. To address this, we decompose the light transport into local and non-local effects. Local appearance changes are modeled using learnable zonal harmonics for diffuse radiance transfer. Unlike spherical harmonics, zonal harmonics are highly efficient to rotate under articulation. This allows us to learn diffuse radiance transfer in a local coordinate frame, which disentangles the local radiance transfer from the articulation of the body. To account for non-local appearance changes, we introduce a shadow network that predicts shadows given precomputed incoming irradiance on a base mesh. This facilitates the learning of non-local shadowing between the body parts. Finally, we use a deferred shading approach to model specular radiance transfer and better capture reflections and highlights such as eye glints. We demonstrate that our approach successfully models both the local and non-local light transport required for relightable full-body avatars, with a superior generalization ability under novel illumination conditions and unseen poses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。