arXiv:2412.09844cs.CV2024-12被引 4

快速防御扩散模型伪造人脸,1秒内完成且不依赖图像特化优化

Real-time Identity Defenses against Malicious Personalization of Diffusion Models

  • 单次前向传播生成对抗扰动,跳过逐图优化
  • 在A100上仅需0.12秒,比领先方法快4400倍
  • 适配手机等边缘设备,可抵御压缩净化等攻击

个性化生成扩散模型能基于少量参考肖像生成高度逼真的图像,可能引发身份复制的社会、伦理与法律风险。现有防御机制依赖计算量大的图像定制化对抗扰动,难以实际部署。本研究提出实时身份防护器(RID),一种神经网络,可通过一次前向传播生成对抗扰动,无需针对每张图像进行优化。RID实现前所未有的效率:在单块NVIDIA A100 80G GPU上仅需0.12秒(比领先方法快4400倍),在标准Intel i9 CPU上每图像仅需1.1秒,适用于智能手机等边缘设备。尽管高效,RID在视觉与量化基准上仍表现出良好保护性能,有效缓解身份复制风险。分析显示,其扰动虽具传统防御效果,但性质不同于自然噪声(如高斯噪声)。为增强鲁棒性,我们扩展出集成框架,融合多个预训练文本到图像扩散模型,确保对黑盒攻击及后处理技术(如图像压缩、净化)的抵抗能力。该模型有望在保护肖像权方面发挥关键作用,防止非法与不道德使用。

原文摘要 · Abstract (English)

Personalized generative diffusion models, capable of synthesizing highly realistic images based on a few reference portraits, may pose substantial social, ethical, and legal risks via identity replication. Existing defense mechanisms rely on computationally intensive adversarial perturbations tailored to individual images, rendering them impractical for real-world deployment. This study introduces the Real-time Identity Defender (RID), a neural network designed to generate adversarial perturbations through a single forward pass, bypassing the need for image-specific optimization. RID achieves unprecedented efficiency, with defense times as low as 0.12 seconds on a single NVIDIA A100 80G GPU (4,400 times faster than leading methods) and 1.1 seconds per image on a standard Intel i9 CPU, making it suitable for edge devices such as smartphones. Despite its efficiency, RID achieves promising protection performance across visual and quantitative benchmarks, effectively mitigating identity replication risks. Our analysis reveals that RID's perturbations mimic the efficacy of traditional defenses while exhibiting properties distinct from natural noise, such as Gaussian perturbations. To enhance robustness, we extend RID into an ensemble framework that integrates multiple pre-trained text-to-image diffusion models, ensuring resilience against black-box attacks and post-processing techniques, including image compression and purification. Our model is envisioned to play a crucial role in safeguarding portrait rights, thereby preventing illegal and unethical uses.

扩散模型身份防护实时防御对抗扰动

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。