研究发现CNN对真实与伪造图像的内部表示方式不同,可为检测伪造图像提供新思路。
Do CNNs Internally Represent Real and Fake Images Differently? A Hidden-Layer Analysis

- 通过对比真实与生成图像在深层激活模式上的差异,检验其内部表征是否不同。
- 伪造图像引发的神经元激活模式显著异于真实图像,且不只由图像退化导致。
- 结果适用于多种模型与数据集,适合关注图像伪造检测的研究者参考。
伪造/合成图像日益泛滥,但卷积神经网络(CNN)是否以相同方式处理真实与伪造图像仍不明确。本文检验了这一假设:即使语义内容一致,伪造图像也会引发不同的隐藏层激活模式。在场景识别任务中,使用训练好的CNN模型提取密集层激活,并结合神经符号方法为选定神经元分配语义标签。对每个真实测试图像,利用基于Stable Diffusion变体的物体标签引导图文生成和图像到图像生成方法生成对应伪造图像。随后对成对的真实-伪造激活模式进行统计比较。额外实验涵盖另一数据集、不同CNN架构、生成模型及JPEG/模糊退化分析,评估结果鲁棒性。结果显示,伪造图像确实诱发不同的隐藏神经元激活,且这种差异无法仅由简单图像退化解释。总体表明,在某些设置下,真实与伪造图像在CNN隐藏层激活行为上存在差异,为后续利用该差异提升伪造图像检测提供了可能。
原文摘要 · Abstract (English)
Fake/synthetic images are increasingly prevalent, but it remains unclear whether Convolutional Neural Networks (CNNs) process real and fake images in the same internal manner. This work examines the hypothesis that CNNs represent real and fake images differently, such that fake images induce different hidden-layer activation patterns even when semantic content is preserved. The hypothesis is evaluated in scene recognition settings using trained CNN models. Dense-layer activations are extracted, and neurosymbolic methods assign semantic labels to selected neurons. For each real test image, corresponding fake images are generated with similar semantic content using object-label-guided text-to-image and image-to-image generation based on Stable Diffusion variants. Paired real-fake activation patterns are then compared statistically. Additional experiments with another dataset, CNN architecture, generative model, and JPEG/blur degradation analysis assess robustness. Results suggest that fake images evoke different hidden-neuron activations, and these differences are not explained only by simple image degradation. Overall, the findings indicate that real and fake images differ in CNN hidden-layer activation behavior at least in some settings, which opens the door for follow-up work on making use of this different behavior to improve fake image detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。