用涂鸦标注提升图像篡改定位精度,降低标注成本。
Beyond Fully Supervised Pixel Annotations: Scribble-Driven Weakly-Supervised Framework for Image Manipulation Localization
- 用涂鸦标注代替像素级标注,构建首个涂鸦驱动的弱监督框架。
- 在主流数据集上超越全监督方法,尤其在分布外场景表现更优。
- 适合需要低成本标注且追求高精度篡改检测的研究者。
近年来,基于深度学习的图像篡改定位(IML)方法取得了显著进展,但通常依赖大规模像素级标注数据集。为解决高质量标注获取困难的问题,部分弱监督方法采用图像级标签进行篡改区域分割,但因监督信号不足,性能仍受限。本文探索一种新的弱监督形式——涂鸦标注,提升了标注效率与检测性能。我们重新标注了主流IML数据集,构建首个基于涂鸦的IML数据集(Sc-IML),并提出首个涂鸦驱动的弱监督IML框架。具体而言,采用带结构一致性损失的自监督训练,使模型在多尺度和增强输入下保持预测一致性;设计先验感知特征调制模块(PFMM),动态融合篡改与真实区域先验信息以增强特征判别力;引入门控自适应融合模块(GAFM),通过门控机制调节特征融合中的信息流,引导模型关注潜在篡改区域;最后提出置信度感知熵最小化损失(${\mathcal{L}}_{ {CEM }}$),根据模型不确定性动态正则化弱标注或未标注区域的预测,有效抑制不可靠预测。实验表明,该方法在分布内与分布外场景下的平均性能均优于现有全监督方法。
原文摘要 · Abstract (English)
Deep learning-based image manipulation localization (IML) methods have achieved remarkable performance in recent years, but typically rely on large-scale pixel-level annotated datasets. To address the challenge of acquiring high-quality annotations, some recent weakly supervised methods utilize image-level labels to segment manipulated regions. However, the performance is still limited due to insufficient supervision signals. In this study, we explore a form of weak supervision that improves the annotation efficiency and detection performance, namely scribble annotation supervision. We re-annotate mainstream IML datasets with scribble labels and propose the first scribble-based IML (Sc-IML) dataset. Additionally, we propose the first scribble-based weakly supervised IML framework. Specifically, we employ self-supervised training with a structural consistency loss to encourage the model to produce consistent predictions under multi-scale and augmented inputs. In addition, we propose a prior-aware feature modulation module (PFMM) that adaptively integrates prior information from both manipulated and authentic regions for dynamic feature adjustment, further enhancing feature discriminability and prediction consistency in complex scenes. We also propose a gated adaptive fusion module (GAFM) that utilizes gating mechanisms to regulate information flow during feature fusion, guiding the model toward emphasizing potential manipulated regions. Finally, we propose a confidence-aware entropy minimization loss (${\mathcal{L}}_{ {CEM }}$). This loss dynamically regularizes predictions in weakly annotated or unlabeled regions based on model uncertainty, effectively suppressing unreliable predictions. Experimental results show that our method outperforms existing fully supervised approaches in terms of average performance both in-distribution and out-of-distribution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。