用网页数据自动生成高质量标注,解决图像篡改定位的数据稀缺难题。
Webly-Supervised Image Manipulation Localization via Category-Aware Auto-Annotation
- 通过类别感知去噪框架自动标注网页伪造图像,降低人工标注成本。
- 构建包含24.6万张图的MIMLv2数据集,规模超现有数据集120倍。
- 适合图像安全、AI检测方向研究者,可提升篡改定位模型性能。
利用网络上大量手工伪造图像和由辅助任务生成的自动标注,缓解图像篡改定位中高质量标注数据稀缺的问题。提出CAAAv2自标注框架,基于类别感知与先验特征去噪,显著降低任务复杂度;设计QES质量评估指标,筛选低质量标注。结合二者构建MIMLv2数据集,含246,212张手动伪造图像及像素级掩码标注,规模超过现有手工数据集IMD20的120倍。引入Object Jitter技术生成高质量篡改痕迹以增强训练。基于此构建Web-IML模型,利用网络规模监督实现高效学习。实验表明,该方法大幅缓解数据稀缺问题,在多个真实伪造基准上显著提升模型性能,相比前序最优模型SparseViT,平均IoU提升21.6点,性能提升达31%。代码与数据集将公开于https://github.com/qcf-568/MIML。
原文摘要 · Abstract (English)
Images manipulated by image editing tools can mislead viewers and pose significant risks to social security. However, accurately localizing manipulated image regions remains challenging due to the severe scarcity of high-quality annotated data, which is laborious to create. To address this, we propose a novel approach that mitigates data scarcity by leveraging readily available web data. We utilize a large collection of manually forged images from the web, as well as automatically generated annotations derived from a simpler auxiliary task, constrained image manipulation localization.Specifically, we introduce CAAAv2, a novel auto-annotation framework that operates on a category-aware, prior-feature-denoising paradigm that notably reduces task complexity. To further ensure annotation reliability, we propose QES, a novel metric that filters out low-quality annotations. Combining CAAAv2 and QES, we construct MIMLv2, a large-scale, diverse, and high-quality dataset containing 246,212 manually forged images with pixel-level mask annotations. This is over 120 times larger than existing handcrafted datasets like IMD20. Additionally, we introduce Object Jitter, a technique that further enhances model training by generating high-quality manipulation artifacts. Building on these advances, we develop Web-IML, a new model designed to effectively leverage web-scale supervision for the task of image manipulation localization. Extensive experiments demonstrate that our approach substantially alleviates the data scarcity problem and significantly improves the performance of various models on multiple real-world forgery benchmarks. With the proposed web supervision, our Web-IML achieves a striking performance gain of 31% and surpasses the previous state-of-the-art SparseViT by 21.6 average IoU points. The dataset and code will be released at https://github.com/qcf-568/MIML.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。