提出新方法破解扩散模型的隐私保护噪声，揭示其脆弱性
CAT: Contrastive Adversarial Training for Evaluating the Robustness of Protective Perturbations in Latent Diffusion Models
- 用轻量适配器设计对抗训练，动态攻击保护噪声
- 实验显示保护噪声在90%以上样本中失效
- 适合关注生成模型安全与隐私防护的研究者
潜在扩散模型在图像生成任务中表现卓越，但使用未经授权数据进行定制会严重侵犯数据所有者的隐私与知识产权。为防御此类滥用，已有研究通过引入人眼不可察觉的噪声（即对抗样例）作为保护扰动，使模型无法有效学习定制数据。本文首次通过定性和定量实验揭示：该保护机制的核心在于扭曲模型的潜在表示。为此，我们提出对比对抗训练（CAT），利用轻量级适配器作为自适应攻击手段，验证了现有保护方法的脆弱性。大量实验表明，CAT显著削弱了保护扰动的有效性，在超过90%的测试样本中使其失效。研究呼吁学界重新评估并改进现有保护机制的鲁棒性。代码已开源于https://github.com/senp98/CAT。
原文摘要 · Abstract (English)
Latent diffusion models have recently demonstrated superior capabilities in many downstream image synthesis tasks. However, customization of latent diffusion models using unauthorized data can severely compromise the privacy and intellectual property rights of data owners. Adversarial examples as protective perturbations have been developed to defend against unauthorized data usage by introducing imperceptible noise to customization samples, preventing diffusion models from effectively learning them. In this paper, we first reveal that the primary reason adversarial examples are effective as protective perturbations in latent diffusion models is the distortion of their latent representations, as demonstrated through qualitative and quantitative experiments. We then propose the Contrastive Adversarial Training (CAT) utilizing lightweight adapters as an adaptive attack against these protection methods, highlighting their lack of robustness. Extensive experiments demonstrate that our CAT method significantly reduces the effectiveness of protective perturbations in customization, urging the community to reconsider and improve the robustness of existing protective perturbations. The code is available at https://github.com/senp98/CAT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。