用更少的高斯混合形状实现实时逼真人脸建模
RGBAvatar: Reduced Gaussian Blendshapes for Online Modeling of Head Avatars
- 用MLP将3DMM参数映射为精简的混合形状权重
- 每秒处理630张图像，重建质量达当前最佳
- 支持视频流实时建模，效果接近离线处理
我们提出减少高斯混合形状的人脸虚拟形象重建方法（RGBAvatar），可在实时条件下完成逼真且可动画化的人脸建模。与以往依赖3D可变形模型（3DMM）线性基的高斯混合形状方法不同，本方法通过MLP将追踪到的3DMM参数映射为精简的混合形状权重，得到紧凑的基函数集合。该学习到的紧凑基组合能有效捕捉个体关键面部细节，不依赖3DMM的固定基权重，从而提升重建质量和效率。为加速重建过程，我们设计了一种新型颜色初始化估计方法和批并行高斯光栅化流程，实现每秒约630张图像的训练吞吐量。此外，提出局部-全局采样策略，支持直接在线重建，可实时接收视频流并即时生成模型，性能媲美离线设置。源代码已公开于https://github.com/gapszju/RGBAvatar。
原文摘要 · Abstract (English)
We present Reduced Gaussian Blendshapes Avatar (RGBAvatar), a method for reconstructing photorealistic, animatable head avatars at speeds sufficient for on-the-fly reconstruction. Unlike prior approaches that utilize linear bases from 3D morphable models (3DMM) to model Gaussian blendshapes, our method maps tracked 3DMM parameters into reduced blendshape weights with an MLP, leading to a compact set of blendshape bases. The learned compact base composition effectively captures essential facial details for specific individuals, and does not rely on the fixed base composition weights of 3DMM, leading to enhanced reconstruction quality and higher efficiency. To further expedite the reconstruction process, we develop a novel color initialization estimation method and a batch-parallel Gaussian rasterization process, achieving state-of-the-art quality with training throughput of about 630 images per second. Moreover, we propose a local-global sampling strategy that enables direct on-the-fly reconstruction, immediately reconstructing the model as video streams in real time while achieving quality comparable to offline settings. Our source code is available at https://github.com/gapszju/RGBAvatar.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。