新方法RADAR可精准识别扩散模型篡改的图像区域。
Towards Reliable Identification of Diffusion-based Image Manipulations
- 融合多模态特征与对比损失，定位篡改区域
- 在28种扩散模型上均表现优异，泛化性强
- 适合图像真实性检测与数字取证研究者
改变面部表情、手势或背景细节可能显著改变图像含义。近年来，扩散模型大幅提升图像编辑质量，也带来滥用风险。因此，可靠识别真实图像中的篡改区域成为重要任务，但不断涌现的新编辑工具持续挑战现有方法。为此，我们提出一种名为RADAR的新方法，用于可靠识别修复区域（ReliAble iDentification of inpainted AReas）。RADAR基于现有基础模型，融合不同图像模态特征，并引入辅助对比损失，有效分离被篡改图像块。实验表明，该方法显著提升检测准确率与对多种扩散模型的泛化能力。为支持真实评估，我们构建了新的基准BBC-PAIR，包含由28种扩散模型生成的篡改图像。结果表明，RADAR在已见和未见模型上的检测与定位性能均超越当前最优方法。代码、数据与模型将公开于https://alex-costanzino.github.io/radar/。
原文摘要 · Abstract (English)
Changing facial expressions, gestures, or background details may dramatically alter the meaning conveyed by an image. Notably, recent advances in diffusion models greatly improve the quality of image manipulation while also opening the door to misuse. Identifying changes made to authentic images, thus, becomes an important task, constantly challenged by new diffusion-based editing tools. To this end, we propose a novel approach for ReliAble iDentification of inpainted AReas (RADAR). RADAR builds on existing foundation models and combines features from different image modalities. It also incorporates an auxiliary contrastive loss that helps to isolate manipulated image patches. We demonstrate these techniques to significantly improve both the accuracy of our method and its generalisation to a large number of diffusion models. To support realistic evaluation, we further introduce BBC-PAIR, a new comprehensive benchmark, with images tampered by 28 diffusion models. Our experiments show that RADAR achieves excellent results, outperforming the state-of-the-art in detecting and localising image edits made by both seen and unseen diffusion models. Our code, data and models will be publicly available at https://alex-costanzino.github.io/radar/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。