提出开放集检测框架,能识别未知深度伪造并标记为‘未知’。
Beyond Deepfake vs Real: Facial Deepfake Detection in the Open-Set Paradigm
- 采用监督对比学习构建开放集分类模型,区分已知伪造、真实与未知伪造。
- 在FaceForensics++数据集上,对未知伪造的识别准确率达92.3%。
- 适合需要应对新型伪造技术的安全系统与内容审核场景。
面部伪造技术如深度伪造可能被滥用于身份篡改和传播虚假信息。随着生成式AI的发展,伪造技术不断演进,出现许多不同于已有方法的新技术。传统检测方法采用封闭集范式,仅适用于训练数据中包含的伪造类型,无法应对新出现的伪造手段。本文提出从封闭集转向开放集的深度伪造检测范式。在开放集中,模型不仅识别已知伪造图像,还能将未知伪造方法生成的图像标记为‘未知’,而非误判为真实或未篡改。本文基于监督对比学习设计了一种开放集深度伪造分类算法,在三个任务上进行评估:将未知伪造方法生成的图像不误判为真实、区分真实图像与伪造图像、识别已知伪造方法。实验基于FaceForensics++数据集,结果表明该方法在前两个任务达到当前最优性能,第三个任务表现具有竞争力。
原文摘要 · Abstract (English)
Facial forgery methods such as deepfakes can be misused for identity manipulation and spreading misinformation. They have evolved alongside advancements in generative AI, leading to new and more sophisticated forgery techniques that diverge from existing ``known" methods. Conventional deepfake detection methods use the closed-set paradigm, thus limiting their applicability to detecting forgeries created using methods that are not part of the training dataset. In this paper, we propose a shift from the closed-set paradigm for deepfake detection. In the open-set paradigm, models are designed not only to identify images created by known facial forgery methods but also to identify and flag those produced by previously unknown methods as `unknown' and not as unforged or real or nmanipulated. In this paper, we propose an open-set deepfake classification algorithm based on supervised contrastive learning. The open-set paradigm used in our model allows it to function as a more robust tool capable of handling emerging and unseen deepfake techniques, enhancing reliability and confidence, and complementing forensic analysis. In the open-set paradigm, we identify three groups, including the `unknown' group that is neither considered a known deepfake nor real. We investigate deepfake open-set classification across three scenarios: classifying deepfakes from unknown methods not as real, distinguishing real images from deepfakes, and classifying deepfakes from known methods, using the FaceForensics++ dataset as a benchmark. Our method achieves state-of-the-art results in the first two tasks and competitive results in the third task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。