弱监督图像篡改定位新方法,提升边缘识别与精度。
Context-Aware Weakly Supervised Image Manipulation Localization with SAM Refinement
- 引入上下文感知边界模块,捕捉篡改区域边缘特征
- 结合CAM与SAM生成更精准的篡改定位图,跨数据集表现优异
- 仅需图像级标签训练,适合缺乏像素标注的场景
恶意图像篡改带来社会风险,亟需高效检测方法。现有方法多依赖全监督,需耗时的像素级标注。为此,我们提出一种弱监督图像篡改定位框架,仅需图像级二值标签。针对现有方法忽略边缘信息的问题,设计上下文感知边界定位(CABL)模块,聚合边界特征并学习上下文不一致性以定位篡改区域。同时,融合类激活图(CAM)与分隔一切模型(SAM),提出CAM引导的SAM精修(CGSR)模块,生成更准确的定位图。整体采用双分支Transformer-CNN架构,在多个数据集上实现卓越定位性能。
原文摘要 · Abstract (English)
Malicious image manipulation poses societal risks, increasing the importance of effective image manipulation detection methods. Recent approaches in image manipulation detection have largely been driven by fully supervised approaches, which require labor-intensive pixel-level annotations. Thus, it is essential to explore weakly supervised image manipulation localization methods that only require image-level binary labels for training. However, existing weakly supervised image manipulation methods overlook the importance of edge information for accurate localization, leading to suboptimal localization performance. To address this, we propose a Context-Aware Boundary Localization (CABL) module to aggregate boundary features and learn context-inconsistency for localizing manipulated areas. Furthermore, by leveraging Class Activation Mapping (CAM) and Segment Anything Model (SAM), we introduce the CAM-Guided SAM Refinement (CGSR) module to generate more accurate manipulation localization maps. By integrating two modules, we present a novel weakly supervised framework based on a dual-branch Transformer-CNN architecture. Our method achieves outstanding localization performance across multiple datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。