arXiv:2505.15385cs.CVcs.GR2025-05International Conf…被引 10

让虚拟人像实时表达面部表情、肢体动作和手势,细节更逼真。

EVA: Expressive Virtual Avatars from Multi-view Videos

  • 分层建模:用模板几何+3D高斯外观分离控制身体与面部表现
  • 从多视角视频中精准追踪动作与表情,实现高保真还原
  • 适合需要精细控制的VR/游戏/影视数字人应用

随着神经渲染和动作捕捉技术的进步,人类虚拟形象建模已取得显著进展,广泛应用于虚拟现实、增强现实、远程通信及游戏、影视、医疗等领域。然而,现有方法因面部表情与身体动作表征纠缠,难以实现完整、真实且可自由控制的虚拟人像。本文提出表达性虚拟人像(EVA),一种针对特定演员的全可控、高度表达性的全身虚拟人像框架,可在实时条件下生成高保真、逼真的视觉效果,并支持对面部表情、身体动作和手部手势的独立控制。具体而言,该方法采用双层模型结构:表达性模板几何层与3D高斯外观层。首先,设计了一种粗到精优化的表达性模板追踪算法,从多视角视频中准确恢复身体动作、面部表情与非刚性形变参数。其次,提出一种解耦的3D高斯外观模型,通过两个独立专用模块分别建模身体与面部外观,有效分离两者表观特征。实验表明,EVA在渲染质量与表达力方面均超越现有最佳方法,验证了其在构建全身虚拟人像方面的有效性。本工作推动了完全可驱动数字人模型的发展,实现了对人类几何与外观的高度忠实复现。

原文摘要 · Abstract (English)

With recent advancements in neural rendering and motion capture algorithms, remarkable progress has been made in photorealistic human avatar modeling, unlocking immense potential for applications in virtual reality, augmented reality, remote communication, and industries such as gaming, film, and medicine. However, existing methods fail to provide complete, faithful, and expressive control over human avatars due to their entangled representation of facial expressions and body movements. In this work, we introduce Expressive Virtual Avatars (EVA), an actor-specific, fully controllable, and expressive human avatar framework that achieves high-fidelity, lifelike renderings in real time while enabling independent control of facial expressions, body movements, and hand gestures. Specifically, our approach designs the human avatar as a two-layer model: an expressive template geometry layer and a 3D Gaussian appearance layer. First, we present an expressive template tracking algorithm that leverages coarse-to-fine optimization to accurately recover body motions, facial expressions, and non-rigid deformation parameters from multi-view videos. Next, we propose a novel decoupled 3D Gaussian appearance model designed to effectively disentangle body and facial appearance. Unlike unified Gaussian estimation approaches, our method employs two specialized and independent modules to model the body and face separately. Experimental results demonstrate that EVA surpasses state-of-the-art methods in terms of rendering quality and expressiveness, validating its effectiveness in creating full-body avatars. This work represents a significant advancement towards fully drivable digital human models, enabling the creation of lifelike digital avatars that faithfully replicate human geometry and appearance.

虚拟人像3D高斯表情控制多视角重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。