首次系统追踪多阶段生成图像的篡改路径,识别每一步使用的模型。
Modelship Attribution: Tracing Multi-Stage Manipulations Across Generative Models
- 构建多阶段篡改流程,模拟真实世界中的迭代编辑。
- 提出MAT模型,准确识别83,700张图像中5个阶段的生成模型。
- 适合做内容溯源、AI造假检测的研究者和安全团队使用。
随着生成技术日益普及,真实图像常被不同个体通过多种工具进行多轮修改。当前检测与溯源方法在单阶段篡改中表现良好,但在复杂现实场景下的多阶段篡改中效果不佳。本文首次系统建模这一挑战,提出“Modelship Attribution”任务:通过识别参与的生成模型并重建编辑序列,追溯图像演化过程。我们使用StyleMapGAN、DiffSwap和FacePartsSwap三个模型对同一图像的不同区域进行连续修改,构建了首个包含83,700张图像(16,740×5)的Modelship数据集。由于后期编辑会覆盖早期模型痕迹,研究重点从提取混合指纹转向刻画各模型的独特编辑模式。为此,我们提出专为该任务设计的Modelship Attribution Transformer(MAT),可有效识别复杂多阶段篡改工作流中各模型的贡献。大量实验与对比分析表明,该方法在多个指标上显著优于现有方法,且通过全面消融实验验证了其有效性。
原文摘要 · Abstract (English)
As generative techniques become increasingly accessible, authentic visuals are frequently subjected to iterative alterations by various individuals employing a variety of tools. Currently, to avoid misinformation and ensure accountability, a lot of research on detection and attribution is emerging. Although these methods demonstrate promise in single-stage manipulation scenarios, they fall short when addressing complex real-world iterative manipulation. In this paper, we are the first, to the best of our knowledge, to systematically model this real-world challenge and introduce a novel method to solve it. We define a task called "Modelship Attribution", which aims to trace the evolution of manipulated images by identifying the generative models involved and reconstructing the sequence of edits they performed. To realistically simulate this scenario, we utilize three generative models, StyleMapGAN, DiffSwap, and FacePartsSwap, that sequentially modify distinct regions of the same image. This process leads to the creation of the first modelship dataset, comprising 83,700 images (16,740 images*5). Given that later edits often overwrite the fingerprints of earlier models, the focus shifts from extracting blended fingerprints to characterizing each model's distinctive editing patterns. To tackle this challenge, we introduce the modelship attribution transformer (MAT), a purpose-built framework designed to effectively recognize and attribute the contributions of various models within complex, multi-stage manipulation workflows. Through extensive experiments and comparative analysis with other related methods, our results, including comprehensive ablation studies, demonstrate that the proposed approach is a highly effective solution for modelship attribution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。