用语言提示提升假脸检测泛化能力,让模型像人一样推理真假。
AuthGuard: Generalizable Deepfake Detection via Language Guidance
- 融合图文对比学习与语言提示,让模型理解逻辑矛盾和感知异常。
- 在DFDC和DF40上分别提升6.15%和16.68%的AUC,跨域检测更准。
- 适合需要可解释性、应对新型伪造的技术团队使用。
现有深度伪造检测方法难以应对不断演化的新型伪造技术,因其依赖训练中学习到的统计特征,这些特征常与特定生成过程绑定,无法代表新出现的伪造样本。我们提出通过引入语言引导来增强检测泛化能力,结合人类常识推理(如识别逻辑矛盾、感知异常)与统计线索。为此,我们训练一个专家级深度伪造视觉编码器,采用判别分类与图像-文本对比学习相结合的方式,文本由通用多模态大模型(MLLMs)通过少样本提示生成。该编码器能从像素分布中提取可被语言描述的常识性伪造痕迹与统计伪造特征。为进一步提升鲁棒性,我们在视觉-语言对比学习中融入数据不确定性学习,减轻图像-文本监督中的噪声影响。该编码器可无缝对接大语言模型(LLM),实现更通用、可解释的检测。所提出的框架AuthGuard在分布内与分布外设置下均达到顶尖性能,在DFDC数据集上提升AUC 6.15%,在DF40上提升16.68%;同时在DDVQA数据集上,推理能力提升24.69%。
原文摘要 · Abstract (English)
Existing deepfake detection techniques struggle to keep-up with the ever-evolving novel, unseen forgeries methods. This limitation stems from their reliance on statistical artifacts learned during training, which are often tied to specific generation processes that may not be representative of samples from new, unseen deepfake generation methods encountered at test time. We propose that incorporating language guidance can improve deepfake detection generalization by integrating human-like commonsense reasoning -- such as recognizing logical inconsistencies and perceptual anomalies -- alongside statistical cues. To achieve this, we train an expert deepfake vision encoder by combining discriminative classification with image-text contrastive learning, where the text is generated by generalist MLLMs using few-shot prompting. This allows the encoder to extract both language-describable, commonsense deepfake artifacts and statistical forgery artifacts from pixel-level distributions. To further enhance robustness, we integrate data uncertainty learning into vision-language contrastive learning, mitigating noise in image-text supervision. Our expert vision encoder seamlessly interfaces with an LLM, further enabling more generalized and interpretable deepfake detection while also boosting accuracy. The resulting framework, AuthGuard, achieves state-of-the-art deepfake detection accuracy in both in-distribution and out-of-distribution settings, achieving AUC gains of 6.15% on the DFDC dataset and 16.68% on the DF40 dataset. Additionally, AuthGuard significantly enhances deepfake reasoning, improving performance by 24.69% on the DDVQA dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。