用图神经网络生成3D高斯点，让头像更逼真且存储仅10MB。
GraphAvatar: Compact Head Avatars with GNN-Generated 3D Gaussians
- 用GNN从追踪的网格生成3D高斯属性，替代存储大量点云。
- 存储仅需10MB，同时保持高质量渲染和实时性能。
- 适合需要轻量化头像的VR/AR应用，尤其关注存储效率者。
从任意视角渲染逼真的头部虚拟形象对虚拟现实等应用至关重要。尽管基于神经辐射场（NeRF）的方法能取得良好效果，但存在保真度和效率不足的问题。近期采用3D高斯泼溅（3DGS）的方法虽提升了渲染质量和实时性，但仍需大量存储空间。本文提出GraphAvatar，利用图神经网络（GNN）生成头部虚拟形象的3D高斯点。具体地，训练几何与外观GNN，从追踪的网格中生成3D高斯属性，从而仅需存储GNN模型而非高斯点，显著降低存储开销至10MB。为缓解面部追踪误差的影响，引入图引导优化模块，在训练中优化追踪参数。此外，设计3D感知增强器用于后处理，进一步提升渲染质量。通过全面实验验证，GraphAvatar在视觉保真度和存储消耗上均优于现有方法。消融研究揭示了渲染质量与模型大小之间的权衡。代码将公开于：https://github.com/ucwxb/GraphAvatar。
原文摘要 · Abstract (English)
Rendering photorealistic head avatars from arbitrary viewpoints is crucial for various applications like virtual reality. Although previous methods based on Neural Radiance Fields (NeRF) can achieve impressive results, they lack fidelity and efficiency. Recent methods using 3D Gaussian Splatting (3DGS) have improved rendering quality and real-time performance but still require significant storage overhead. In this paper, we introduce a method called GraphAvatar that utilizes Graph Neural Networks (GNN) to generate 3D Gaussians for the head avatar. Specifically, GraphAvatar trains a geometric GNN and an appearance GNN to generate the attributes of the 3D Gaussians from the tracked mesh. Therefore, our method can store the GNN models instead of the 3D Gaussians, significantly reducing the storage overhead to just 10MB. To reduce the impact of face-tracking errors, we also present a novel graph-guided optimization module to refine face-tracking parameters during training. Finally, we introduce a 3D-aware enhancer for post-processing to enhance the rendering quality. We conduct comprehensive experiments to demonstrate the advantages of GraphAvatar, surpassing existing methods in visual fidelity and storage consumption. The ablation study sheds light on the trade-offs between rendering quality and model size. The code will be released at: https://github.com/ucwxb/GraphAvatar
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。