单张无姿态图像秒级生成高保真3D头像,无需优化
PanoLAM: Large Avatar Model for Gaussian Full-Head Synthesis from One-shot Unposed Image
- 单次前向传播完成全头高斯建模,跳过耗时优化
- 用合成数据训练,实现无真实3D头像数据下的高质量生成
- 分阶段细化生成+双路特征融合,提升重建精度
我们提出一种前馈式框架,仅需单张无姿态图像即可生成高斯全头模型。与以往依赖耗时的GAN反演和测试时优化的工作不同,本框架可在一次前向传播中完成重建,显著加速推理过程。为缓解大规模3D头部资产缺失问题,我们基于训练好的3D GAN构建了大规模合成数据集,并仅使用合成数据训练模型。为实现高效高保真生成,我们设计了粗到精的高斯头像生成流程:利用FLAME模型稀疏点通过Transformer块提取图像特征并重建粗略形状,再进行密集化以实现高保真重建。为进一步利用预训练3D GAN中的先验知识,我们提出双分支架构,有效融合结构化球面三平面特征与非结构化点特征,提升高斯头像重建效果。实验表明,该框架在性能上优于现有方法。
原文摘要 · Abstract (English)
We present a feed-forward framework for Gaussian full-head synthesis from a single unposed image. Unlike previous work that relies on time-consuming GAN inversion and test-time optimization, our framework can reconstruct the Gaussian full-head model given a single unposed image in a single forward pass. This enables fast reconstruction and rendering during inference. To mitigate the lack of large-scale 3D head assets, we propose a large-scale synthetic dataset from trained 3D GANs and train our framework using only synthetic data. For efficient high-fidelity generation, we introduce a coarse-to-fine Gaussian head generation pipeline, where sparse points from the FLAME model interact with the image features by transformer blocks for feature extraction and coarse shape reconstruction, which are then densified for high-fidelity reconstruction. To fully leverage the prior knowledge residing in pretrained 3D GANs for effective reconstruction, we propose a dual-branch framework that effectively aggregates the structured spherical triplane feature and unstructured point-based features for more effective Gaussian head reconstruction. Experimental results show the effectiveness of our framework towards existing work. Project page at: https://panolam.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。