用前视图生成完整3D虚拟人,补全后脑部细节。
AvatarBack: Back-Head Generation for Complete 3D Avatars from Front-View Images
- 基于生成先验和自适应对齐,从少量正面图像合成后视伪图。
- 在NeRSemble和K-hairstyle数据集上显著提升后脑几何与视觉质量。
- 适合需要完整可动画3D虚拟人的应用,如虚拟主播、数字人建模。
最近的高斯点阵技术大幅提升了头部虚拟人的重建质量,通过将3D虚拟人表示为一组3D高斯分布实现高保真面部建模。然而,现有方法主要依赖正面图像,导致后脑区域重建较差,出现几何不一致、结构模糊等问题,限制了整体真实感。为此,我们提出AvatarBack,一个专为完整3D高斯虚拟人重建设计的即插即用框架,明确建模缺失的后脑区域。该框架包含两项核心技术:主体特定生成器(SSG)和自适应空间对齐策略(ASA)。前者利用生成先验,从稀疏正面输入生成身份一致且合理的后视伪图像,提供多视角监督;后者通过训练时优化的可学习变换矩阵,精确对齐合成视图与3D高斯表示,有效解决固有的姿态与坐标偏差问题。在NeRSemble和K-hairstyle数据集上的大量实验表明,采用几何、光度及GPT-4o感知指标评估,AvatarBack显著提升后脑重建质量,同时保持正面精度。此外,重建的虚拟人具备多样运动下的视觉一致性,且完全可动画化。
原文摘要 · Abstract (English)
Recent advances in Gaussian Splatting have significantly boosted the reconstruction of head avatars, enabling high-quality facial modeling by representing an 3D avatar as a collection of 3D Gaussians. However, existing methods predominantly rely on frontal-view images, leaving the back-head poorly constructed. This leads to geometric inconsistencies, structural blurring, and reduced realism in the rear regions, ultimately limiting the fidelity of reconstructed avatars. To address this challenge, we propose AvatarBack, a novel plug-and-play framework specifically designed to reconstruct complete and consistent 3D Gaussian avatars by explicitly modeling the missing back-head regions. AvatarBack integrates two core technical innovations,i.e., the Subject-specific Generator (SSG) and the Adaptive Spatial Alignment Strategy (ASA). The former leverages a generative prior to synthesize identity-consistent, plausible back-view pseudo-images from sparse frontal inputs, providing robust multi-view supervision. To achieve precise geometric alignment between these synthetic views and the 3D Gaussian representation, the later employs learnable transformation matrices optimized during training, effectively resolving inherent pose and coordinate discrepancies. Extensive experiments on NeRSemble and K-hairstyle datasets, evaluated using geometric, photometric, and GPT-4o-based perceptual metrics, demonstrate that AvatarBack significantly enhances back-head reconstruction quality while preserving frontal fidelity. Moreover, the reconstructed avatars maintain consistent visual realism under diverse motions and remain fully animatable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。