通过3D人脸建模实现表情动作动画中面部身份一致
Identity-Preserving Pose-Guided Character Animation via Facial Landmarks Transformation
- 用3D可变形模型转换2D关键点，对齐参考人脸几何结构
- 在舞蹈等动态场景下，面部身份一致性提升显著
- 适合需要高保真人脸重建的动画生成任务
在跳舞等复杂动态场景中，生成姿态引导的图像到视频角色动画并保持面部身份真实感仍具挑战性。现有方法常因驱动视频提取的面部关键点与参考图像的面部结构不匹配，导致面部失真。为此，本文提出面部关键点转换（FLT）方法，利用3D可变形模型将2D关键点映射为3D人脸，调整其与参考身份对齐后，再映射回2D关键点，用于指导图像到视频生成。该方法有效提升了生成视频与参考图像之间的面部一致性。实验表明，FLT显著改善了姿态引导角色动画模型的面部身份保持能力。
原文摘要 · Abstract (English)
Creating realistic pose-guided image-to-video character animations while preserving facial identity remains challenging, especially in complex and dynamic scenarios such as dancing, where precise identity consistency is crucial. Existing methods frequently encounter difficulties maintaining facial coherence due to misalignments between facial landmarks extracted from driving videos that provide head pose and expression cues and the facial geometry of the reference images. To address this limitation, we introduce the Facial Landmarks Transformation (FLT) method, which leverages a 3D Morphable Model to address this limitation. FLT converts 2D landmarks into a 3D face model, adjusts the 3D face model to align with the reference identity, and then transforms them back into 2D landmarks to guide the image-to-video generation process. This approach ensures accurate alignment with the reference facial geometry, enhancing the consistency between generated videos and reference images. Experimental results demonstrate that FLT effectively preserves facial identity, significantly improving pose-guided character animation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。