arXiv:2602.06122cs.CV2026-02中稿 · 3DV 2026被引 1

用3D生成先验提升低质人脸视频,生成逼真可动的高清3D头像。

From Blurry to Believable: Enhancing Low-quality Talking Heads with 3D Generative Priors

  • 通过动态感知的3D反演,融合预训练3D生成模型先验增强低质输入。
  • 在动态表情和多视角下生成细节丰富、时序一致的高清3D头像。
  • 适合需要高质量可动画3D人脸的应用,如虚拟主播、元宇宙角色。

高保真、可动画的3D说话头像对沉浸式应用至关重要,但常受低质量图像或视频源限制,导致3D重建效果差。本文提出SuperHead框架,用于增强低分辨率、可动画的3D人脸头像。核心挑战在于合成高质量几何与纹理,同时确保动画过程中的3D与时间一致性,并保持主体身份。尽管图像、视频和3D超分辨率(SR)技术已有进展,现有方法难以处理动态3D输入。SuperHead利用预训练3D生成模型的丰富先验,通过新颖的动态感知3D反演方案优化生成模型的潜在表示,生成超分辨率的3D高斯泼溅(3DGS)头像模型,并进一步绑定到参数化头模型(如FLAME)以实现动画。反演过程联合监督于从多种表情和摄像机视角采集的稀疏上采样2D人脸渲染图与对应深度图,以保证动态面部运动下的真实感。实验表明,SuperHead在动态运动下生成具有精细面部细节的头像,视觉质量显著优于基线方法。

原文摘要 · Abstract (English)

Creating high-fidelity, animatable 3D talking heads is crucial for immersive applications, yet often hindered by the prevalence of low-quality image or video sources, which yield poor 3D reconstructions. In this paper, we introduce SuperHead, a novel framework for enhancing low-resolution, animatable 3D head avatars. The core challenge lies in synthesizing high-quality geometry and textures, while ensuring both 3D and temporal consistency during animation and preserving subject identity. Despite recent progress in image, video and 3D-based super-resolution (SR), existing SR techniques are ill-equipped to handle dynamic 3D inputs. To address this, SuperHead leverages the rich priors from pre-trained 3D generative models via a novel dynamics-aware 3D inversion scheme. This process optimizes the latent representation of the generative model to produce a super-resolved 3D Gaussian Splatting (3DGS) head model, which is subsequently rigged to an underlying parametric head model (e.g., FLAME) for animation. The inversion is jointly supervised using a sparse collection of upscaled 2D face renderings and corresponding depth maps, captured from diverse facial expressions and camera viewpoints, to ensure realism under dynamic facial motions. Experiments demonstrate that SuperHead generates avatars with fine-grained facial details under dynamic motions, significantly outperforming baseline methods in visual quality.

3D生成超分辨率人脸动画3DGS

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。