用扩散模型反向生成伪造人脸,提升检测器跨域泛化能力
DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion

- 将检测模型特征注入预训练扩散模型,反向重建源脸与目标脸
- 在多个数据集上显著提升检测器对未知伪造手法的识别率
- 无需额外参数,适配各类检测模型,适合安全防护场景
深度伪造技术快速发展,人脸换脸已高度逼真,引发恶意内容滥用担忧。现有方法常因面部篡改形式多样而难以泛化到未见领域。本文重新审视生成过程,发现伪造图像始终包含源身份与目标身份信息,而真实人脸则保持身份一致。基于此,提出DiffusionFake——一种即插即用框架,通过将检测模型提取的特征注入冻结的Stable Diffusion模型,引导其重建对应的源脸与目标脸。该受控重建迫使检测网络学习源/目标相关特征,从而获得更丰富、解耦的表示,增强对未知伪造的鲁棒性。大量实验表明,DiffusionFake可显著提升多种检测架构的跨域泛化性能,且推理时无额外参数。代码已开源。
原文摘要 · Abstract (English)
The rapid progress of Deepfake technology has made face swapping highly realistic, raising concerns about the malicious use of fabricated facial content. Existing methods often struggle to generalize to unseen domains due to the diverse nature of facial manipulations. In this paper, we revisit the generation process and identify a universal principle: Deepfake images inherently contain information from both source and target identities, while genuine faces maintain a consistent identity. Building upon this insight, we introduce DiffusionFake, a novel plug-and-play framework that reverses the generative process of face forgeries to enhance the generalization of detection models. DiffusionFake achieves this by injecting the features extracted by the detection model into a frozen pre-trained Stable Diffusion model, compelling it to reconstruct the corresponding target and source images. This guided reconstruction process constrains the detection network to capture the source and target related features to facilitate the reconstruction, thereby learning rich and disentangled representations that are more resilient to unseen forgeries. Extensive experiments demonstrate that DiffusionFake significantly improves cross-domain generalization of various detector architectures without introducing additional parameters during inference. Our Codes are available in https://github.com/skJack/DiffusionFake.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。