arXiv:2608.04722cs.CVcs.GR2026-08中稿 · SCA 2026

分离建模面部与手势,实现高保真上半身虚拟人动画

Multi-View Face and Gesture Animation with Dynamic Gaussians

论文配图:Multi-View Face and Gesture Animation with Dynamic Gaussians
图 1 · 摘自论文原文
  • 分模块建模面部与手部,融合参数化躯干网格
  • 在多视角下生成高质量动态渲染结果,支持新视角呈现
  • 构建首个含表情与手势的多视角上半身动捕数据集

生成具有真实上半身动作的逼真3D虚拟人仍具挑战。现有方法或忽视手部动作,或重建全身但难以保持面部细节与手部姿态精度。为此,我们提出MVFGA,一种多视角一致的上半身虚拟人生成流程。该方法分别建模面部与手部,并融合至参数化上半身网格,从而精准捕捉细微表情与手势。随后将3D高斯分布点映射到重建网格,实现从新视角的高质量渲染。此外,我们构建了MVFGA-MoCap数据集,包含受控表情序列、多样手势及自由交流动作。实验表明,MVFGA在上半身动画中显著优于基线方法,生成结果在视觉真实性和动作保真度上均表现优异。

原文摘要 · Abstract (English)

Creating photorealistic 3D human avatars with realistic upper-body motion remains challenging. Existing approaches either focus on the head and overlook hand gestures, or reconstruct the full body but fail to preserve fine-grained facial fidelity and hand pose accuracy. As a result, current methods struggle to capture the subtle dynamics of facial expressions and hand gestures that are crucial for natural human communication. While methods based on full-body parametric models enable avatar reconstruction from monocular or multi-view inputs, they often lack accurate facial animation and detailed hand articulation. To address these limitations, we propose MVFGA, a novel multi-view-consistent pipeline for generating realistic upper-body avatars. Our approach models the face and hands separately and fuses them with a parametric upper-body mesh model, enabling the capture of fine-grained facial expressions and hand poses for accurate upper-body avatar reconstruction. We then splat 3D Gaussians onto the obtained mesh, enabling high-quality rendering of dynamic avatars from novel viewpoints. Furthermore, we introduce MVFGA-MoCap, a multi-view upper-body motion capture dataset featuring controlled facial expression sequences, diverse hand gestures, and free-form communication. Experiments show that MVFGA generates visually realistic avatars with high-fidelity facial expressions and hand motions, outperforming baselines for upper-body avatar animation. Project page: https://dfki-av.github.io/MVFGA/

虚拟人生成动作捕捉3D高斯

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。