arXiv:2412.02421cs.CV2024-12

用一生影像建模可动3D人脸,实现跨年龄真实还原与动画。

TimeWalker: Personalized Neural Space for Lifelong Head Avatars

  • 通过动态神经基底混合学习跨龄人脸的个性化表征。
  • 在多个生命阶段实现高保真重建与表情驱动动画。
  • 适合数字人、虚拟角色长期演化研究者使用。

我们提出TimeWalker,一种全新框架,用于建模个体全生命周期的逼真、完整3D人脸形象。不同于现有仅基于瞬时影像或短视频的人脸建模方法,TimeWalker从个体不同人生阶段的非结构化数据中构建其完整身份表征,实现任意时间点的高质量重建与动画。核心是新型神经参数化模型,能解耦形状、表情与外观随年龄变化的特征。方法包含两方面:(1)基于标准空间平均头像与一组神经头基底的加性组合建模身份;提出动态神经基底混合模块(Dynamo),根据共享与特异性特征自适应调整基底数量与权重;(2)扩展高斯点阵表示为动态2D高斯点阵(DNA-2DGS),利用参数模型先验控制2D定向平面高斯盘的运动与旋转,实现表情变形的同时保持渲染真实感。大量实验表明,TimeWalker可在解耦维度上实现跨年龄真人级重建与动画,轻松实现个性化的‘时光穿越’。

原文摘要 · Abstract (English)

We present TimeWalker, a novel framework that models realistic, full-scale 3D head avatars of a person on lifelong scale. Unlike current human head avatar pipelines that capture identity at the momentary level(e.g., instant photography or short videos), TimeWalker constructs a person's comprehensive identity from unstructured data collection over his/her various life stages, offering a paradigm to achieve full reconstruction and animation of that person at different moments of life. At the heart of TimeWalker's success is a novel neural parametric model that learns personalized representation with the disentanglement of shape, expression, and appearance across ages. Central to our methodology are the concepts of two aspects: (1) We track back to the principle of modeling a person's identity in an additive combination of average head representation in the canonical space, and moment-specific head attribute representations driven from a set of neural head basis. To learn the set of head basis that could represent the comprehensive head variations in a compact manner, we propose a Dynamic Neural Basis-Blending Module (Dynamo). It dynamically adjusts the number and blend weights of neural head bases, according to both shared and specific traits of the target person over ages. (2) Dynamic 2D Gaussian Splatting (DNA-2DGS), an extension of Gaussian splatting representation, to model head motion deformations like facial expressions without losing the realism of rendering and reconstruction. DNA-2DGS includes a set of controllable 2D oriented planar Gaussian disks that utilize the priors from parametric model, and move/rotate with the change of expression. Through extensive experimental evaluations, we show TimeWalker's ability to reconstruct and animate avatars across decoupled dimensions with realistic rendering effects, demonstrating a way to achieve personalized 'time traveling' in a breeze.

3D人脸跨龄建模神经渲染动态生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。