让手机实时生成高质量3D虚拟头像，兼顾速度与画质。
PrismAvatar: Real-time animated 3D neural head avatars on edge devices
- 用网格+可变形神经辐射场混合建模，兼顾细节与效率。
- 在移动端实现60帧/秒运行，内存占用低，效果接近高端设备。
- 适合移动AR/VR、实时视频通话等边缘计算场景。
我们提出PrismAvatar：一种专为资源受限的边缘设备设计的3D头像模型，可在训练时利用神经体渲染优势，推理时实现实时动画与渲染。通过将带骨骼的棱柱格网与3D可变形头像模型结合，采用混合渲染策略同时重建基于网格的头部和未被3DMM覆盖区域的可变形NeRF。随后将可变形NeRF蒸馏为带骨骼的网格与神经纹理，使其能在传统三角形渲染管线中高效动画与渲染。该模型在移动端可实现60帧/秒的运行速度且内存占用低，其训练结果在桌面设备上与当前顶尖3D头像模型质量相当。
原文摘要 · Abstract (English)
We present PrismAvatar: a 3D head avatar model which is designed specifically to enable real-time animation and rendering on resource-constrained edge devices, while still enjoying the benefits of neural volumetric rendering at training time. By integrating a rigged prism lattice with a 3D morphable head model, we use a hybrid rendering model to simultaneously reconstruct a mesh-based head and a deformable NeRF model for regions not represented by the 3DMM. We then distill the deformable NeRF into a rigged mesh and neural textures, which can be animated and rendered efficiently within the constraints of the traditional triangle rendering pipeline. In addition to running at 60 fps with low memory usage on mobile devices, we find that our trained models have comparable quality to state-of-the-art 3D avatar models on desktop devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。