通过对齐语义与伪造特征空间,提升图像伪造检测精度。
Semantic Discrepancy-aware Detector for Image Forgery Identification
- 设计语义令牌采样模块,减少无关特征干扰。
- 基于视觉重建学习概念级伪造差异,捕捉细粒度伪造痕迹。
- 适合关注图像真实性验证的研究者与应用开发者。
随着图像生成技术的快速发展,确保数字媒体可信性,稳健的伪造检测变得愈发重要。研究表明,预训练模型所学的语义概念对识别伪造图像至关重要,但伪造与语义概念空间之间的错位会限制模型性能。为此,我们提出一种新的语义差异感知检测器(SDD),利用重构学习在细粒度视觉层面对齐两个空间。通过挖掘预训练视觉语言模型中的概念知识,我们特别设计了语义令牌采样模块,以缓解由与伪造痕迹和语义概念均无关的特征引起的空间偏移。进一步提出基于视觉重构范式的概念级伪造差异学习模块,增强视觉语义概念与伪造痕迹间的交互,有效捕捉在概念引导下的差异。最后,低层级伪造特征增强模块融合学习到的概念级伪造差异,最小化冗余伪造信息。在两个标准图像伪造数据集上的实验表明,所提SDD优于现有方法。代码已公开于https://github.com/wzy1111111/SSD。
原文摘要 · Abstract (English)
With the rapid advancement of image generation techniques, robust forgery detection has become increasingly imperative to ensure the trustworthiness of digital media. Recent research indicates that the learned semantic concepts of pre-trained models are critical for identifying fake images. However, the misalignment between the forgery and semantic concept spaces hinders the model's forgery detection performance. To address this problem, we propose a novel Semantic Discrepancy-aware Detector (SDD) that leverages reconstruction learning to align the two spaces at a fine-grained visual level. By exploiting the conceptual knowledge embedded in the pre-trained vision language model, we specifically design a semantic token sampling module to mitigate the space shifts caused by features irrelevant to both forgery traces and semantic concepts. A concept-level forgery discrepancy learning module, built upon a visual reconstruction paradigm, is proposed to strengthen the interaction between visual semantic concepts and forgery traces, effectively capturing discrepancies under the concepts' guidance. Finally, the low-level forgery feature enhancemer integrates the learned concept level forgery discrepancies to minimize redundant forgery information. Experiments conducted on two standard image forgery datasets demonstrate the efficacy of the proposed SDD, which achieves superior results compared to existing methods. The code is available at https://github.com/wzy1111111/SSD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。