arXiv:2411.19117cs.CV2024-11被引 14

用视觉基础模型检测生成图像,不需训练也能发现深度伪造。

Understanding and Improving Training-Free AI-Generated Image Detections with Vision Foundation Models

  • 利用模型对图像扰动的敏感性差异,无训练识别真假图像。
  • 高斯模糊比噪声更有效,尤其在人脸图像上表现更好。
  • 提出新方法MINDER,解决不同噪声类型带来的偏差问题。

生成模型的快速发展带来了深度伪造等严重风险,如人脸合成与编辑。传统方法依赖训练分类器并借助多种特征提取提升泛化能力。而无需训练的检测方法则直接利用视觉基础模型的统计特性,通过区分真实与虚假图像的嵌入响应来实现检测。当前领先的训练自由方法RIGID基于DINOv2对图像空间扰动的敏感性,发现虚假图像嵌入比真实图像更敏感。我们系统研究了模型主干、扰动类型和数据集对检测性能的影响,发现性能与模型鲁棒性密切相关:自监督学习(SSL)模型提供更可靠表征。高斯噪声在通用物体上表现良好,但在人脸图像上效果较差;而高斯模糊因能捕捉频率伪影更有效。为此,我们提出对比模糊(Contrastive Blur)提升人脸检测性能,并设计MINDER(MINimum distance DetEctoR),缓解噪声类型偏差,平衡跨域表现。本工作不仅带来性能提升,还深化了对模型鲁棒性用于深度伪造检测的理解。

原文摘要 · Abstract (English)

The rapid advancement of generative models has introduced serious risks, including deepfake techniques for facial synthesis and editing. Traditional approaches rely on training classifiers and enhancing generalizability through various feature extraction techniques. Meanwhile, training-free detection methods address issues like limited data and overfitting by directly leveraging statistical properties from vision foundation models to distinguish between real and fake images. The current leading training-free approach, RIGID, utilizes DINOv2 sensitivity to perturbations in image space for detecting fake images, with fake image embeddings exhibiting greater sensitivity than those of real images. This observation prompts us to investigate how detection performance varies across model backbones, perturbation types, and datasets. Our experiments reveal that detection performance is closely linked to model robustness, with self-supervised (SSL) models providing more reliable representations. While Gaussian noise effectively detects general objects, it performs worse on facial images, whereas Gaussian blur is more effective due to potential frequency artifacts. To further improve detection, we introduce Contrastive Blur, which enhances performance on facial images, and MINDER (MINimum distance DetEctoR), which addresses noise type bias, balancing performance across domains. Beyond performance gains, our work offers valuable insights for both the generative and detection communities, contributing to a deeper understanding of model robustness property utilized for deepfake detection.

深度伪造无训练检测视觉模型图像生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。