arXiv:2501.02376cs.CV2025-01ICML被引 1

识别文生图扩散模型生成图像的原始来源,防止滥用。

Origin Identification for Text-Guided Image-to-Image Diffusion Models

  • 提出线性变换方法,统一不同模型生成图像与原图的特征距离。
  • 在跨模型测试中比现有方法高出31.6%的mAP指标。
  • 构建首个通用数据集OriPID,支持多模型验证。

文生图扩散模型能根据文本提示精准修改图像,但易被滥用于传播虚假信息、侵犯版权和逃避内容追踪。为此,我们提出图像源识别任务(ID²),旨在从生成图像中还原原始图像。传统相似度匹配方法因不同扩散模型间视觉差异而失效。为此,我们构建了首个通用数据集OriPID,包含大量原始图像与引导提示。理论证明:预训练变分自编码器(VAE)嵌入空间中存在一个线性变换,可最小化生成样本与原始图像间的距离;且该变换具备跨模型泛化能力。实验表明,该方法在跨模型测试中显著优于现有方法,平均精度提升31.6% mAP,且适用于多种扩散模型。

原文摘要 · Abstract (English)

Text-guided image-to-image diffusion models excel in translating images based on textual prompts, allowing for precise and creative visual modifications. However, such a powerful technique can be misused for spreading misinformation, infringing on copyrights, and evading content tracing. This motivates us to introduce the task of origin IDentification for text-guided Image-to-image Diffusion models (ID$^2$), aiming to retrieve the original image of a given translated query. A straightforward solution to ID$^2$ involves training a specialized deep embedding model to extract and compare features from both query and reference images. However, due to visual discrepancy across generations produced by different diffusion models, this similarity-based approach fails when training on images from one model and testing on those from another, limiting its effectiveness in real-world applications. To solve this challenge of the proposed ID$^2$ task, we contribute the first dataset and a theoretically guaranteed method, both emphasizing generalizability. The curated dataset, OriPID, contains abundant Origins and guided Prompts, which can be used to train and test potential IDentification models across various diffusion models. In the method section, we first prove the existence of a linear transformation that minimizes the distance between the pre-trained Variational Autoencoder (VAE) embeddings of generated samples and their origins. Subsequently, it is demonstrated that such a simple linear transformation can be generalized across different diffusion models. Experimental results show that the proposed method achieves satisfying generalization performance, significantly surpassing similarity-based methods ($+31.6\%$ mAP), even those with generalization designs. The project is available at https://id2icml.github.io.

图像溯源扩散模型特征对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。