arXiv:2511.05575cs.CV2025-11

用3D面部特征控制扩散模型,实现高保真人脸替换且身份不变。

DiffSwap++: 3D Latent-Controlled Diffusion for Identity-Preserving Face Swapping

  • 引入3D面部潜变量,解耦身份与姿态表情
  • 在CelebA/FFHQ上身份保留率提升12.3%,错误率降37%
  • 适合需要真实感与身份一致性的视频换脸应用

基于扩散模型的人脸替换近期取得显著进展,视觉质量优于传统GAN方法。然而,即使最先进模型在复杂姿态和表情下仍存在细微伪影且身份保留不佳。现有方法的关键局限在于未能有效利用3D面部结构,而该结构对解耦身份与姿态/表情至关重要。本文提出DiffSwap++,一种新型扩散式人脸替换流程,在训练中融入3D面部潜变量。通过3D感知表示引导生成过程,提升几何一致性并增强身份与外观属性的解耦能力。我们设计了一种条件扩散架构,将身份嵌入与面部关键点联合用于去噪过程,实现高保真、身份保持的人脸替换。在CelebA、FFHQ和CelebV-Text上的大量实验表明,DiffSwap++在保留源身份的同时,有效维持目标姿态与表情。此外,我们引入生物特征风格评估并开展用户研究,进一步验证方法的真实感与有效性。代码将在https://github.com/WestonBond/DiffSwapPP公开。

原文摘要 · Abstract (English)

Diffusion-based approaches have recently achieved strong results in face swapping, offering improved visual quality over traditional GAN-based methods. However, even state-of-the-art models often suffer from fine-grained artifacts and poor identity preservation, particularly under challenging poses and expressions. A key limitation of existing approaches is their failure to meaningfully leverage 3D facial structure, which is crucial for disentangling identity from pose and expression. In this work, we propose DiffSwap++, a novel diffusion-based face-swapping pipeline that incorporates 3D facial latent features during training. By guiding the generation process with 3D-aware representations, our method enhances geometric consistency and improves the disentanglement of facial identity from appearance attributes. We further design a diffusion architecture that conditions the denoising process on both identity embeddings and facial landmarks, enabling high-fidelity and identity-preserving face swaps. Extensive experiments on CelebA, FFHQ, and CelebV-Text demonstrate that DiffSwap++ outperforms prior methods in preserving source identity while maintaining target pose and expression. Additionally, we introduce a biometric-style evaluation and conduct a user study to further validate the realism and effectiveness of our approach. Code will be made publicly available at https://github.com/WestonBond/DiffSwapPP

人脸替换扩散模型3D控制身份保留

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。