arXiv:2605.04702cs.CVcs.AI2026-05

解决文本生成视频时人脸姿态变化导致的身份失真问题

FaithfulFaces: Pose-Faithful Facial Identity Preservation for Text-to-Video Generation

论文配图:FaithfulFaces: Pose-Faithful Facial Identity Preservation for Text-to-Video Generation
图 1 · 摘自论文原文
  • 用共享姿态字典和姿态不变性约束对齐多视角人脸
  • 在大幅表情变化和遮挡下仍保持高身份一致性
  • 适合需要稳定人脸特征的影视生成与虚拟形象应用

身份保持的文本到视频生成(IPT2V)使用户能以一致的人脸身份生成多样且富有想象力的视频。尽管已有进展,现有方法在大幅人脸姿态变化或遮挡情况下仍常出现身份失真。本文提出FaithfulFaces,一种姿态忠实的人脸身份保持学习框架,用于提升复杂动态场景下的IPT2V表现。核心是姿态共享身份对齐器,通过姿态共享字典和姿态变化-身份不变性约束,跨视角精炼并对齐人脸姿态。通过显式欧拉角嵌入将单视角输入映射至全局人脸姿态表示,提供姿态忠实的人脸先验,引导生成模型实现鲁棒的身份保持生成。特别地,我们构建了一个高质量视频数据集,涵盖显著的人脸姿态多样性。大量实验表明,FaithfulFaces达到当前最优性能,在姿态变化和遮挡条件下仍保持优异的身份一致性和结构清晰度。

原文摘要 · Abstract (English)

Identity-preserving text-to-video generation (IPT2V) empowers users to produce diverse and imaginative videos with consistent human facial identity. Despite recent progress, existing methods often suffer from significant identity distortion under large facial pose variations or facial occlusions. In this paper, we propose \textit{FaithfulFaces}, a pose-faithful facial identity preservation learning framework to improve IPT2V in complex dynamic scenes. The key of FaithfulFaces is a pose-shared identity aligner that refines and aligns facial poses across distinct views via a pose-shared dictionary and a pose variation-identity invariance constraint. By mapping single-view inputs into a global facial pose representation with explicit Euler angle embeddings, FaithfulFaces provides a pose-faithful facial prior that guides generative foundations toward robust identity-preserving generation. In particular, we develop a specialized pipeline to curate a high-quality video dataset featuring substantial facial pose diversity. Extensive experiments demonstrate that FaithfulFaces achieves state-of-the-art performance, maintaining superior identity consistency and structural clarity even as pose changes and occlusions occur.

文本生成视频人脸身份保持姿态鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。