通过分析图像风格差异,无需原始人脸即可检测换脸伪造视频。
De-Fake: Style based Anomaly Deepfake Detection
- 基于图像风格特征而非面部关键点或像素不一致进行检测
- 在多个数据集上对不同换脸方法均表现出高准确率
- 无需访问真实人脸,天然保护隐私,适合实际应用
检测涉及人脸替换的深度伪造(deepfake)是一个重大挑战,尤其在现实场景中,任何人都可使用免费工具轻松实现换脸。现有方法依赖面部关键点或像素级特征不一致,但在无缝融合源人脸的目标图像或视频中表现不佳。换脸技术被广泛用于传播虚假信息、损害声誉、操纵舆论、制作非自愿性亲密内容(NCID),以及生成儿童性虐待材料(CSAM)。公众人物也常成为目标,相关伪造内容在社交媒体上广泛传播。另一个难点是构建涵盖多种变化的数据集,因训练需大量数据,引发个人面部数据的隐私担忧。本文提出一种基于风格差异的检测方法,无需访问真实人脸即可有效识别换脸图像。我们在多个数据集和换脸方法上进行了全面评估,验证了该方法在多样化场景下的有效性。SafeVision提供了一种可靠且可扩展的隐私保护式换脸检测方案,特别适用于复杂现实应用。据我们所知,SafeVision是首个利用风格特征并具备内在隐私保护能力的深度伪造检测方法。
原文摘要 · Abstract (English)
Detecting deepfakes involving face-swaps presents a significant challenge, particularly in real-world scenarios where anyone can perform face-swapping with freely available tools and apps without any technical knowledge. Existing deepfake detection methods rely on facial landmarks or inconsistencies in pixel-level features and often struggle with face-swap deepfakes, where the source face is seamlessly blended into the target image or video. The prevalence of face-swap is evident in everyday life, where it is used to spread false information, damage reputations, manipulate political opinions, create non-consensual intimate deepfakes (NCID), and exploit children by enabling the creation of child sexual abuse material (CSAM). Even prominent public figures are not immune to its impact, with numerous deepfakes of them circulating widely across social media platforms. Another challenge faced by deepfake detection methods is the creation of datasets that encompass a wide range of variations, as training models require substantial amounts of data. This raises privacy concerns, particularly regarding the processing and storage of personal facial data, which could lead to unauthorized access or misuse. Our key idea is to identify these style discrepancies to detect face-swapped images effectively without accessing the real facial image. We perform comprehensive evaluations using multiple datasets and face-swapping methods, which showcases the effectiveness of SafeVision in detecting face-swap deepfakes across diverse scenarios. SafeVision offers a reliable and scalable solution for detecting face-swaps in a privacy preserving manner, making it particularly effective in challenging real-world applications. To the best of our knowledge, SafeVision is the first deepfake detection using style features while providing inherent privacy protection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。