用扩散模型实现高保真视频换脸，保持时序稳定性和细节真实
HiFiVFS: High Fidelity Video Face Swapping
- 基于SVD扩散模型，利用时序先验提升视频连贯性
- 引入细粒度属性模块，更好保留光影与妆容等细节
- 适合影视制作、虚拟形象生成等对画质要求高的场景
人脸换脸旨在将源图像的身份特征与目标图像的属性结合。现有方法多聚焦于图像级换脸，处理视频时逐帧独立处理，难以保证时间稳定性。从模型角度看，换脸技术正从生成对抗网络（GANs）转向扩散模型（DMs），因后者具备更强的生成能力。当前基于扩散模型的方法常采用修复（inpainting）技术，难以保持如光照、妆容等细粒度属性。为此，我们提出高保真视频换脸框架HiFiVFS，充分利用稳定视频扩散模型（Stable Video Diffusion, SVD）的强生成能力与时序先验。构建细粒度属性模块，通过身份去敏感化和对抗学习提取解耦的身份与细粒度属性特征，并引入精细化身份注入机制进一步增强身份相似性。大量实验表明，该方法在视频换脸任务中达到当前最优（SOTA）表现，无论定性还是定量指标均领先。
原文摘要 · Abstract (English)
Face swapping aims to generate results that combine the identity from the source with attributes from the target. Existing methods primarily focus on image-based face swapping. When processing videos, each frame is handled independently, making it difficult to ensure temporal stability. From a model perspective, face swapping is gradually shifting from generative adversarial networks (GANs) to diffusion models (DMs), as DMs have been shown to possess stronger generative capabilities. Current diffusion-based approaches often employ inpainting techniques, which struggle to preserve fine-grained attributes like lighting and makeup. To address these challenges, we propose a high fidelity video face swapping (HiFiVFS) framework, which leverages the strong generative capability and temporal prior of Stable Video Diffusion (SVD). We build a fine-grained attribute module to extract identity-disentangled and fine-grained attribute features through identity desensitization and adversarial learning. Additionally, We introduce detailed identity injection to further enhance identity similarity. Extensive experiments demonstrate that our method achieves state-of-the-art (SOTA) in video face swapping, both qualitatively and quantitatively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。