用扩散模型生成修复区域掩码,精准定位图像篡改边界。
InpDiffusion: Image Inpainting Localization via Conditional Diffusion Models
- 将修复定位转为条件掩码生成,结合语义与边缘信息逐步优化。
- 在多个数据集上显著超越现有方法,对细微篡改边界检测更准确。
- 适合需要高精度图像取证与篡改检测的研究者使用。
随着生成模型如GAN和扩散模型的快速发展,图像修复定位(IIL)的准确性面临挑战。现有方法存在过度自信导致误判、难以识别细微修复边界的问题。为此,我们提出一种新范式:将IIL视为条件掩码生成任务,利用扩散模型实现。InpDiffusion通过引入图像语义条件增强去噪过程,逐步细化预测结果;同时在去噪中融入边缘条件,并设计新型边缘监督策略,提升对修复对象边界的感知能力。该机制平衡了扩散模型的随机采样与边缘监督,有效降低误判风险并避免因过度随机性丢失细节。此外,我们提出双流多尺度特征提取器(DMFE),融合语义与边缘信息以增强特征表达。大量实验表明,InpDiffusion在多个挑战性数据集上显著优于现有最优方法,具备出色的泛化能力和鲁棒性。
原文摘要 · Abstract (English)
As artificial intelligence advances rapidly, particularly with the advent of GANs and diffusion models, the accuracy of Image Inpainting Localization (IIL) has become increasingly challenging. Current IIL methods face two main challenges: a tendency towards overconfidence, leading to incorrect predictions; and difficulty in detecting subtle tampering boundaries in inpainted images. In response, we propose a new paradigm that treats IIL as a conditional mask generation task utilizing diffusion models. Our method, InpDiffusion, utilizes the denoising process enhanced by the integration of image semantic conditions to progressively refine predictions. During denoising, we employ edge conditions and introduce a novel edge supervision strategy to enhance the model's perception of edge details in inpainted objects. Balancing the diffusion model's stochastic sampling with edge supervision of tampered image regions mitigates the risk of incorrect predictions from overconfidence and prevents the loss of subtle boundaries that can result from overly stochastic processes. Furthermore, we propose an innovative Dual-stream Multi-scale Feature Extractor (DMFE) for extracting multi-scale features, enhancing feature representation by considering both semantic and edge conditions of the inpainted images. Extensive experiments across challenging datasets demonstrate that the InpDiffusion significantly outperforms existing state-of-the-art methods in IIL tasks, while also showcasing excellent generalization capabilities and robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。