用语音实时驱动虚拟人面部表情，支持游戏角色动画制作。
Audio2Face-3D: Audio-driven Realistic Facial Animation For Digital Avatars
- 基于语音输入实时生成逼真面部动作
- 开源完整工具链，含训练框架与示例数据集
- 适合游戏开发与虚拟形象创作者使用
语音驱动的面部动画为数字虚拟人动画提供了一种高效解决方案。本文详细介绍了NVIDIA Audio2Face-3D的技术细节，包括数据采集、网络架构、重定向方法、评估指标及应用场景。Audio2Face-3D系统实现了人与交互式虚拟人之间的实时互动，便于游戏角色的面部动画创作。为帮助数字虚拟人创作者和游戏开发者生成真实感面部动画，我们已开源Audio2Face-3D的模型网络、SDK、训练框架及示例数据集。
原文摘要 · Abstract (English)
Audio-driven facial animation presents an effective solution for animating digital avatars. In this paper, we detail the technical aspects of NVIDIA Audio2Face-3D, including data acquisition, network architecture, retargeting methodology, evaluation metrics, and use cases. Audio2Face-3D system enables real-time interaction between human users and interactive avatars, facilitating facial animation authoring for game characters. To assist digital avatar creators and game developers in generating realistic facial animations, we have open-sourced Audio2Face-3D networks, SDK, training framework, and example dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。