提出两类通用伪造痕迹,用伪假数据训练模型实现跨生成器的面部深度伪造检测。
From Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakes
- 将伪造痕迹分为面部不一致和上采样两类,聚焦共性特征。
- 仅用伪假数据训练的分类器在未见生成器上仍具良好泛化能力。
- 适合需要跨模型检测的深度伪造防御场景。
深度伪造检测因AI生成技术迅猛发展而愈发重要。本文探讨如何构建对多数人脸深度伪造有效的通用检测框架。现有方法面临生成器种类繁多、伪造痕迹各异(如光照不一致、色彩错位等)的挑战。是否需逐一学习所有痕迹?显然不可行。核心思路是识别各类深度伪造中更普遍的共性痕迹。我们将其分为两类互补类型:面部不一致痕迹(FIA)与上采样痕迹(USA)。FIA源于生成复杂面部细节时与周围区域均匀性的矛盾;USA则是生成器解码过程中的必然残留。观察发现,几乎所有现有深度伪造均呈现其一或两者。为此,我们提出一种数据级伪假生成框架,仅构造含FIA与USA的假样本,避免引入非通用痕迹。具体地,通过超分辨率模拟USA,设计自融合模块(Blender)对多样面部区域进行图像级自混合以生成FIA。令人意外的是,仅用该伪假数据训练的标准图像分类器,即可在未见过的深度伪造上表现出显著泛化能力。
原文摘要 · Abstract (English)
Detecting deepfakes has been an increasingly important topic, especially given the rapid development of AI generation techniques. In this paper, we ask: How can we build a universal detection framework that is effective for most facial deepfakes? One significant challenge is the wide variety of deepfake generators available, resulting in varying forgery artifacts (e.g., lighting inconsistency, color mismatch, etc). But should we ``teach" the detector to learn all these artifacts separately? It is impossible and impractical to elaborate on them all. So the core idea is to pinpoint the more common and general artifacts across different deepfakes. Accordingly, we categorize deepfake artifacts into two distinct yet complementary types: Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA). FIA arise from the challenge of generating all intricate details, inevitably causing inconsistencies between the complex facial features and relatively uniform surrounding areas. USA, on the other hand, are the inevitable traces left by the generator's decoder during the up-sampling process. This categorization stems from the observation that all existing deepfakes typically exhibit one or both of these artifacts. To achieve this, we propose a new data-level pseudo-fake creation framework that constructs fake samples with only the FIA and USA, without introducing extra less-general artifacts. Specifically, we employ a super-resolution to simulate the USA, while design a Blender module that uses image-level self-blending on diverse facial regions to create the FIA. We surprisingly found that, with this intuitive design, a standard image classifier trained only with our pseudo-fake data can non-trivially generalize well to unseen deepfakes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。