提出针对人脸微调扩散模型的成员推理攻击，可判断人脸是否被用于训练。
Membership Inference Attacks for Face Images Against Fine-Tuned Latent Diffusion Models
- 用生成辅助数据和水印提升攻击性能
- 推理时引导尺度影响显著，长期微调后提示词无影响
- 黑箱环境下对人脸微调模型有效，适合隐私安全研究者
生成图像模型的兴起引发了训练数据大规模使用带来的隐私担忧。本文研究了能否推断一组人脸图像是否被用于微调潜在扩散模型（LDM）。提出一种成员推理攻击（MIA）方法：利用生成的辅助数据训练攻击模型，显著提升性能；引入水印也能增强效果。实验发现，推理时的引导尺度具有显著影响；若模型经过足够长时间的微调，文本提示对攻击结果无显著影响。所提MIA在真实黑箱设置下对人脸微调的LDM表现可行，验证了潜在隐私风险。
原文摘要 · Abstract (English)
The rise of generative image models leads to privacy concerns when it comes to the huge datasets used to train such models. This paper investigates the possibility of inferring if a set of face images was used for fine-tuning a Latent Diffusion Model (LDM). A Membership Inference Attack (MIA) method is presented for this task. Using generated auxiliary data for the training of the attack model leads to significantly better performance, and so does the use of watermarks. The guidance scale used for inference was found to have a significant influence. If a LDM is fine-tuned for long enough, the text prompt used for inference has no significant influence. The proposed MIA is found to be viable in a realistic black-box setup against LDMs fine-tuned on face-images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。