用微小干扰保护人脸隐私，防模型滥用
Privacy Protection in Personalized Diffusion Models via Targeted Cross-Attention Adversarial Attack
- 仅攻击扩散模型的交叉注意力层，精准干扰用户特定特征
- 低噪声下效果优于现有方法，保护隐私更高效
- 适合关注图像生成隐私安全的研究者与开发者
个性化文本到图像扩散模型在生成定制化视觉内容方面需求日益增长，但其被恶意使用时存在重大隐私风险。本文提出一种新型高效对抗攻击方法——概念保护选择性注意力操控（CoPSAM），仅针对扩散模型的交叉注意力层进行攻击。通过在微调过程中添加不可察觉的噪声，最大化用户特定标记与类别特定标记对应的交叉注意力图之间的差异，从而生成对抗样本。在CelebA-HQ人脸数据集子集上的实验表明，该方法性能优于现有方案。定性评估显示：（i）在更低噪声水平下即实现更优保护效果；（ii）有效防止内容被未经授权使用，保护个体身份不被滥用。
原文摘要 · Abstract (English)
The growing demand for customized visual content has led to the rise of personalized text-to-image (T2I) diffusion models. Despite their remarkable potential, they pose significant privacy risk when misused for malicious purposes. In this paper, we propose a novel and efficient adversarial attack method, Concept Protection by Selective Attention Manipulation (CoPSAM) which targets only the cross-attention layers of a T2I diffusion model. For this purpose, we carefully construct an imperceptible noise to be added to clean samples to get their adversarial counterparts. This is obtained during the fine-tuning process by maximizing the discrepancy between the corresponding cross-attention maps of the user-specific token and the class-specific token, respectively. Experimental validation on a subset of CelebA-HQ face images dataset demonstrates that our approach outperforms existing methods. Besides this, our method presents two important advantages derived from the qualitative evaluation: (i) we obtain better protection results for lower noise levels than our competitors; and (ii) we protect the content from unauthorized use thereby protecting the individual's identity from potential misuse.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。