让个性化图像生成模型自带数据溯源能力,防止未经授权使用训练数据。
Towards Reliable Verification of Unauthorized Data Usage in Personalized Text-to-Image Diffusion Models
- 通过优化水印特征使其契合个性化任务,提升模型学习能力。
- 在多个基准数据集上实现90%以上验证准确率,抗干扰能力强。
- 适合关注版权保护与数据安全的AI开发者与研究者。
文本到图像扩散模型正不断拓展生成式AI的应用边界。除生成通用图像外,新提出的个性化技术可对预训练基础模型进行定制,以生成特定主题或风格的图像。这种轻量级方案使开发者能便捷构建专属模型,但也引发担忧:个性化模型是否使用了未经授权的数据?一种有前景的解决方案是主动在生成模型中嵌入数据可追溯性,即数据所有者在发布数据前添加外部涂层(如图像水印或后门触发器),训练后的模型会无意识地在生成结果中重现这些涂层,作为数据使用的证据。然而,我们发现现有涂层在个性化任务中难以被有效学习,导致验证可靠性不足。本文提出SIREN,一种针对黑箱个性化文本到图像扩散模型的新型数据溯源方法。该方法精心优化涂层,使其被模型视为与个性化任务相关的特征,显著提升其可学习性。同时结合人眼感知约束、超球分类技术和假设检验引导的验证方法,增强涂层的隐蔽性与检测准确性。SIREN在多种基准数据集、模型和学习算法上经过充分实验验证,且在真实场景中表现良好,能抵御潜在反制措施。代码已公开。
原文摘要 · Abstract (English)
Text-to-image diffusion models are pushing the boundaries of what generative AI can achieve in our lives. Beyond their ability to generate general images, new personalization techniques have been proposed to customize the pre-trained base models for crafting images with specific themes or styles. Such a lightweight solution, enabling AI practitioners and developers to easily build their own personalized models, also poses a new concern regarding whether the personalized models are trained from unauthorized data. A promising solution is to proactively enable data traceability in generative models, where data owners embed external coatings (e.g., image watermarks or backdoor triggers) onto the datasets before releasing. Later the models trained over such datasets will also learn the coatings and unconsciously reproduce them in the generated mimicries, which can be extracted and used as the data usage evidence. However, we identify the existing coatings cannot be effectively learned in personalization tasks, making the corresponding verification less reliable. In this paper, we introduce SIREN, a novel methodology to proactively trace unauthorized data usage in black-box personalized text-to-image diffusion models. Our approach optimizes the coating in a delicate way to be recognized by the model as a feature relevant to the personalization task, thus significantly improving its learnability. We also utilize a human perceptual-aware constraint, a hypersphere classification technique, and a hypothesis-testing-guided verification method to enhance the stealthiness and detection accuracy of the coating. The effectiveness of SIREN is verified through extensive experiments on a diverse set of benchmark datasets, models, and learning algorithms. SIREN is also effective in various real-world scenarios and evaluated against potential countermeasures. Our code is publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。