arXiv:2602.12381cs.CV2026-02被引 2

用CLIP检测假图,发现它靠的是图像质量与构图线索。

Synthetic Image Detection with CLIP: Understanding and Assessing Predictive Cues

  • 通过对比真实照片与配文匹配的生成图,分析CLIP判断依据。
  • 在GAN数据集上准确率达0.96,在扩散模型数据集上为0.92。
  • 适合关注生成图像质量特征或跨模型泛化能力的研究者。

近期生成模型可产出生动逼真的图像,挑战了照片的可信度。合成图像检测(SID)方法常难以跨数据集和生成模型泛化。尽管CLIP在该任务表现良好,但其决策背后的线索仍不明确。本文将基于CLIP的SID视为实证可解释性问题,而非提出新检测器。我们引入SynthCLIC,将真实照片与其对应、高质量扩散生成的配图文图配对。在SynthCLIC、以GAN为主的基准和外部广泛基准上评估基于CLIP的检测器,对比低层取证CNN、广覆盖生成器检测器和文本引导概念模型。基于CLIP的线性检测器在GAN重基准上达0.96 mAP,SynthCLIC上为0.92,跨家族迁移至CNNSpot时降至0.42 mAP。类内关联显示,高合成得分对应更干净、构图更受控、技术更精良的图像;低得分则关联于混乱拍摄条件和真实图像的来源线索。这些关联分布在多个重叠线索中,且不同训练数据集差异显著。因此,基于CLIP与取证检测器失败方式不同,提供互补证据,而广覆盖生成器训练对稳健的SID至关重要。

原文摘要 · Abstract (English)

Recent generative models produce near-photorealistic images, challenging the trustworthiness of photographs. Synthetic image detection (SID) methods, however, often struggle to generalize across datasets and generative models. CLIP, which embeds images and text in a shared semantic space, performs well at SID, but the cues underlying its decisions remain poorly understood. We therefore study CLIP-based SID as an empirical interpretability problem rather than proposing a new detector. We introduce SynthCLIC, which pairs real photographs with caption-matched, high-quality diffusion-generated counterparts. We evaluate CLIP-based detectors on SynthCLIC, a GAN-heavy benchmark, and a broad external benchmark, and compare them with a low-level forensic CNN, a broad-generator detector, and a text-grounded concept model. CLIP-based linear detectors reach 0.96 mAP on the GAN-heavy benchmark but 0.92 on SynthCLIC, while cross-family transfer to CNNSpot falls to 0.42 mAP. Within-class associations between detector scores and text-derived cue scores show that higher synthetic scores correspond to cleaner, more compositionally controlled, and technically polished images, whereas lower scores correspond to messier capture conditions and provenance cues characteristic of real photographs. These associations are distributed across many overlapping cues, and their profiles differ strongly across training datasets. CLIP-based and forensic detectors therefore fail in different ways and provide complementary evidence, while broad generator coverage appears important for robust SID.

图像检测CLIP生成模型可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。