通过引导模型关注真实图像的分布偏差,提升对未知生成模型图像的检测能力。
Detecting AI-Generated Images via Distributional Deviations from Real Images
- 利用纹理感知掩码机制微调CLIP-ViT,聚焦生成图像的异常分布特征。
- 仅用少量样本微调即达98.2%和94.6%准确率,显著优于现有方法。
- 适合需要跨模型泛化的AI图像检测场景,如社交媒体内容审核。
生成模型的快速发展极大提升了图像质量,引发虚假信息传播与公众信任危机。检测AI生成图像已成为关键挑战,尤其需具备对未见生成模型的泛化能力。现有基于冻结预训练CLIP模型的方法虽具泛化潜力,但仅将其作为基础特征提取器,未能充分挖掘其性能。本文深入分析冻结的CLIP-ViT图像编码器发现,其能在高层抽象特征空间中聚类真实图像,但不具备真实/生成图像区分能力。基于此,提出掩码微调(MPFT)策略,引入纹理感知掩码(TAM)机制,在微调过程中掩蔽包含生成模型特异性模式的纹理区域,迫使模型关注生成图像与真实图像之间的分布偏差。在GenImage与UniversalFakeDetect数据集上的大量实验表明,仅用极少样本微调即可显著超越现有方法,分别达到98.2%与94.6%的平均准确率。
原文摘要 · Abstract (English)
The rapid advancement of generative models has significantly enhanced the quality of AI-generated images, raising concerns about misinformation and the erosion of public trust. Detecting AI-generated images has thus become a critical challenge, particularly in terms of generalizing to unseen generative models. Existing methods using frozen pre-trained CLIP models show promise in generalization but treat the image encoder as a basic feature extractor, failing to fully exploit its potential. In this paper, we perform an in-depth analysis of the frozen CLIP image encoder (CLIP-ViT), revealing that it effectively clusters real images in a high-level, abstract feature space. However, it does not truly possess the ability to distinguish between real and AI-generated images. Based on this analysis, we propose a Masking-based Pre-trained model Fine-Tuning (MPFT) strategy, which introduces a Texture-Aware Masking (TAM) mechanism to mask textured areas containing generative model-specific patterns during fine-tuning. This approach compels CLIP-ViT to attend to the "distributional deviations"from authentic images for AI-generated image detection, thereby achieving enhanced generalization performance. Extensive experiments on the GenImage and UniversalFakeDetect datasets demonstrate that our method, fine-tuned with only a minimal number of images, significantly outperforms existing approaches, achieving up to 98.2% and 94.6% average accuracy on the two datasets, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。