无需训练即可检测生成图像,靠噪声残差聚类区分真伪
Effective Synthetic Image Detection via Noise Residual Clustering
- 通过预训练模型提取图像噪声残差,再用冻结ViT提取多尺度特征
- 仅需少量真实图像初始化聚类中心,无监督实现82.2%平均准确率
- 对扩散模型生成图像检测效果突出,适合快速部署的反伪造场景
生成式人工智能的快速发展使合成图像愈发逼真,带来虚假信息与欺诈等安全威胁。被动且盲目的图像认证检测至关重要。现有方法多依赖大规模标注数据进行有监督训练,成本高且对未知生成模型泛化能力差。为此,我们提出一种无需训练的检测方法:首先使用简单的预训练Noiseprint++模型提取噪声残差指纹;随后利用冻结的视觉变换器(ViT)从该残差中提取多尺度特征,并进行自适应加权融合;仅需少量真实图像样本初始化聚类中心,通过无监督K-Means实现真实与合成图像的区分,无需训练。在四个基准数据集上的广泛评估表明,所提方案平均准确率达82.2%,优于当前最先进检测器,在扩散型合成图像上表现更优。消融实验验证了各模块有效性。源代码将公开于https://github.com/multimediaFor/NoiseCluSID。
原文摘要 · Abstract (English)
The rapid advancement of generative artificial intelligence (AI) has made synthetic images remarkably realistic, posing security threats such as misinformation and fraud. It is significant to detect the synthetic image in the manner of passive and blind image authentication. Most existing detectors rely on supervised training with large labeled datasets, leading to high costs and degraded performance on unknown generative models. To attenuate such deficiencies, we propose a training-free detection method. Specifically, noise residual fingerprints are first extracted by a simple yet effective pre-trained Noiseprint++ model. Then multi-scale features are further extracted from such residual by a frozen Vision Transformer (ViT), followed by adaptive weighted fusion. Only a few real image samples are used needed to initialize the clustering centers for unsupervised K-Means, distinguishing real and synthetic images without training. Extensive evaluations on four benchmark datasets show that our proposed scheme achieves an average accuracy of 82.2%, outperforming the state-of-the-art detectors on generalization ability. Superior performance is gained on the popular diffusion type of synthetic images, and the effectiveness of each module is validated by ablation studies. Source code will be publicly available at https://github.com/multimediaFor/NoiseCluSID.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。