用低成本采集头像，实现高精度可重光照的动态虚拟人
BecomingLit: Relightable Gaussian Avatars with Hybrid Neural Shading
- 基于3D高斯和参数化头部模型，结合表情动态模块驱动
- 融合神经漫反射与解析高光项，支持全频段光照重渲染
- 仅需单目视频即可控制，适合虚拟人、数字孪生应用
我们提出BecomingLit，一种重建可重光照、高分辨率头部虚拟人的新方法，可在交互式帧率下从新视角渲染。为此，我们设计了一种专为面部优化的低成本光照扫描系统，并收集了一个新数据集，包含多位受试者在不同光照和表情下的多视角序列。基于该数据集，我们提出一种基于3D高斯原语的可重光照虚拟人表示，通过参数化头部模型和表情依赖动态模块进行动画驱动。我们引入一种混合神经着色方法，结合神经漫反射BRDF与解析高光项。该方法能从动态光照录制中解耦材质，支持点光源和环境贴图的全频段重光照。此外，虚拟人可轻松由单目视频驱动。我们在新数据集上进行大量实验，结果表明其在重光照和重演方面显著优于现有最先进方法。
原文摘要 · Abstract (English)
We introduce BecomingLit, a novel method for reconstructing relightable, high-resolution head avatars that can be rendered from novel viewpoints at interactive rates. Therefore, we propose a new low-cost light stage capture setup, tailored specifically towards capturing faces. Using this setup, we collect a novel dataset consisting of diverse multi-view sequences of numerous subjects under varying illumination conditions and facial expressions. By leveraging our new dataset, we introduce a new relightable avatar representation based on 3D Gaussian primitives that we animate with a parametric head model and an expression-dependent dynamics module. We propose a new hybrid neural shading approach, combining a neural diffuse BRDF with an analytical specular term. Our method reconstructs disentangled materials from our dynamic light stage recordings and enables all-frequency relighting of our avatars with both point lights and environment maps. In addition, our avatars can easily be animated and controlled from monocular videos. We validate our approach in extensive experiments on our dataset, where we consistently outperform existing state-of-the-art methods in relighting and reenactment by a significant margin.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。