提出首个针对文本生成图像模型版权验证的规避攻击方法。
Towards Dataset Copyright Evasion Attack against Personalized Text-to-Image Diffusion Models

- 通过分析模型特征收敛速度差异,精准识别含水印数据。
- 迭代剔除提示词中的关键标记,定位触发水印的触发词。
- 采用闭式概念消除法移除水印,不损害模型生成能力。
文本到图像(T2I)扩散模型可基于文本提示生成高质量图像。然而,为个性化微调预训练模型时,存在未经授权使用数据集的风险。为此,近期提出了数据集所有权验证(DOV)机制,通过后门技术将水印嵌入微调数据集:正常样本下水印保持隐匿,触发时则输出指定内容。尽管前景广阔,其对抗版权规避攻击(CEA)的鲁棒性尚未被探索。本文研究了攻击者如何绕过此类机制,使基于带水印数据集训练的模型逃避所有权验证。我们分析了现有攻击方法如TPD和T2IShield的局限性——前者因随机性导致效果不稳定,后者在水印以局部图像块形式嵌入时失效。为此,我们提出首个专攻T2I扩散模型中DOV的规避攻击方法CEAT2I,包含三阶段:(1) 基于模型在水印样本上中间特征比训练损失更快收敛的现象,可靠检测水印样本;(2) 迭代剔除检测样本的提示词标记,并监测特征变化,定位触发水印的触发词;(3) 应用闭式概念消除法去除注入的水印。大量实验表明,CEAT2I能有效规避当前最先进的DOV机制,同时保持模型性能。代码已开源:https://github.com/csyufei/CEAT2I。
原文摘要 · Abstract (English)
Text-to-image (T2I) diffusion models enable high-quality image generation conditioned on textual prompts. However, fine-tuning these pre-trained models for personalization raises concerns about unauthorized dataset usage. To address this issue, dataset ownership verification (DOV) has recently been proposed, which embeds watermarks into fine-tuning datasets via backdoor techniques. These watermarks remain dormant on benign samples but produce owner-specified outputs when triggered. Despite its promise, the robustness of DOV against copyright evasion attacks (CEA) remains unexplored. In this paper, we investigate how adversaries can circumvent these mechanisms, enabling models trained on watermarked datasets to bypass ownership verification. We begin by analyzing the limitations of potential attacks achieved by backdoor removal, including TPD and T2IShield. In practice, TPD suffers from inconsistent effectiveness due to randomness, while T2IShield fails when watermarks are embedded as local image patches. To this end, we introduce CEAT2I, the first CEA specifically targeting DOV in T2I diffusion models. CEAT2I consists of three stages: (1) motivated by the observation that T2I models converge faster on watermarked samples with respect to intermediate features rather than training loss, we reliably detect watermarked samples; (2) we iteratively ablate tokens from the prompts of detected samples and monitor feature shifts to identify trigger tokens; and (3) we apply a closed-form concept erasure method to remove the injected watermarks. Extensive experiments demonstrate that CEAT2I effectively evades state-of-the-art DOV mechanisms while preserving model performance. The code is available at https://github.com/csyufei/CEAT2I.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。