用混合表示实现手机端逼真头像实时渲染
Gaussian Pixel Codec Avatars: A Hybrid Representation for Efficient Rendering
- 结合三角网格与3D高斯，分别处理皮肤和毛发等区域
- 在保持照片级真实感的同时，渲染速度媲美传统网格模型
- 适合移动端实时虚拟形象应用，如社交、游戏
我们提出Gaussian Pixel Codec Avatars（GPiCA），一种可从多视角图像生成、并在移动设备上高效渲染的逼真头部虚拟形象。GPiCA采用独特的混合表示：三角网格用于高效建模面部皮肤等表面区域，3D高斯则处理头发、胡须等非表面细节。为此，我们构建统一的可微分渲染管线，将网格作为半透明层嵌入3D高斯点云的体素渲染框架中。通过多视角图像监督训练神经网络，将表情编码解码为3D人脸网格、RGBA纹理及一组3D高斯。三者在统一渲染引擎中同步输出。结果表明，GPiCA在视觉真实度上达到纯高斯方法水平，同时渲染性能接近网格方法。
原文摘要 · Abstract (English)
We present Gaussian Pixel Codec Avatars (GPiCA), photorealistic head avatars that can be generated from multi-view images and efficiently rendered on mobile devices. GPiCA utilizes a unique hybrid representation that combines a triangle mesh and anisotropic 3D Gaussians. This combination maximizes memory and rendering efficiency while maintaining a photorealistic appearance. The triangle mesh is highly efficient in representing surface areas like facial skin, while the 3D Gaussians effectively handle non-surface areas such as hair and beard. To this end, we develop a unified differentiable rendering pipeline that treats the mesh as a semi-transparent layer within the volumetric rendering paradigm of 3D Gaussian Splatting. We train neural networks to decode a facial expression code into three components: a 3D face mesh, an RGBA texture, and a set of 3D Gaussians. These components are rendered simultaneously in a unified rendering engine. The networks are trained using multi-view image supervision. Our results demonstrate that GPiCA achieves the realism of purely Gaussian-based avatars while matching the rendering performance of mesh-based avatars.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。