arXiv:2411.14823cs.CVcs.CR2024-11被引 8

首个统一多任务图像篡改定位的通用模型,解决传统方法泛化差问题。

Omni-IML: Towards Unified Image Manipulation Localization

  • 通过动态选择编码模态、自适应解码器和异常增强模块实现跨任务泛化。
  • 在四个主流篡改定位任务上均达最新水平,单模型性能超越专用模型。
  • 适用于需要多类型篡改检测与解释的现实场景,如媒体审核与取证。

现有图像篡改定位(IML)方法多依赖特定任务设计,仅在目标任务上表现良好,联合训练多种任务时性能显著下降,限制了实际应用。为此,我们提出 Omni-IML,首个面向多样化任务的通用图像篡改定位模型。其通过三个核心组件实现泛化:(1) 模态门控编码器,根据样本自适应选择最优编码模态;(2) 动态权重解码器,根据任务动态调整解码滤波器;(3) 异常增强模块,利用框监督突出篡改区域,促进学习任务无关特征。为支持篡改图像的可解释性,我们构建了 Omni-273k 大规模高质量数据集,包含自然语言描述的篡改痕迹,通过自动链式思维标注技术生成。我们还设计了一个简单有效的解释模块以更好利用这些描述注释。大量实验表明,单一 Omni-IML 模型在四个主要 IML 任务上均达到当前最佳性能,为实际部署提供可行方案,并展示了图像取证领域通用模型的前景。代码与数据集将公开。

原文摘要 · Abstract (English)

Existing Image Manipulation Localization (IML) methods mostly rely heavily on task-specific designs, making them perform well only on the target IML task, while joint training on multiple IML tasks causes significant performance degradation, hindering real applications. To this end, we propose Omni-IML, the first generalist model designed to unify IML across diverse tasks. Specifically, Omni-IML achieves generalization through three key components: (1) a Modal Gate Encoder, which adaptively selects the optimal encoding modality per sample, (2) a Dynamic Weight Decoder, which dynamically adjusts decoder filters to the task at hand, and (3) an Anomaly Enhancement module that leverages box supervision to highlight the tampered regions and facilitate the learning of task-agnostic features. Beyond localization, to support interpretation of the tampered images, we construct Omni-273k, a large high-quality dataset that includes natural language descriptions of tampered artifact. It is annotated through our automatic, chain-of-thoughts annotation technique. We also design a simple-yet-effective interpretation module to better utilize these descriptive annotations. Our extensive experiments show that our single Omni-IML model achieves state-of-the-art performance across all four major IML tasks, providing a valuable solution for practical deployment and a promising direction of generalist models in image forensics. Our code and dataset will be publicly available.

图像取证通用模型篡改定位可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。