现有会员推理攻击在真实场景下失效，研究提出新基准验证其不可靠性。
Real-World Benchmarks Make Membership Inference Attacks Fail on Diffusion Models
- 构建更真实的评估基准CopyMark，支持预训练模型与公平评测
- 实测显示当前攻击方法在真实条件下效果大幅下降
- 警告当前会员推理攻击无法可靠检测扩散模型数据盗用
扩散模型的会员推理攻击（MIAs）被视为训练中未经授权使用数据的潜在证据。这些攻击旨在检测特定图像是否存在于扩散模型的训练数据集中。本研究深入评估了当前最先进的扩散模型会员推理攻击，揭示了现有评估中存在关键缺陷和过度乐观的性能估计。我们提出了CopyMark，一个更符合实际的会员推理攻击基准，具备支持预训练扩散模型、无偏数据集和公平评估流程的特点。通过大量实验，我们证明了当前会员推理方法在更现实条件下有效性显著下降。基于结果，我们警示：在当前状态下，会员推理攻击无法可靠用于识别预训练扩散模型中的未经授权数据使用。据我们所知，这是首个发现扩散模型会员推理攻击性能被高估的研究，并提出了统一的更真实评估基准。代码已开源：https://github.com/caradryanl/CopyMark。
原文摘要 · Abstract (English)
Membership inference attacks (MIAs) on diffusion models have emerged as potential evidence of unauthorized data usage in training pre-trained diffusion models. These attacks aim to detect the presence of specific images in training datasets of diffusion models. Our study delves into the evaluation of state-of-the-art MIAs on diffusion models and reveals critical flaws and overly optimistic performance estimates in existing MIA evaluation. We introduce CopyMark, a more realistic MIA benchmark that distinguishes itself through the support for pre-trained diffusion models, unbiased datasets, and fair evaluation pipelines. Through extensive experiments, we demonstrate that the effectiveness of current MIA methods significantly degrades under these more practical conditions. Based on our results, we alert that MIA, in its current state, is not a reliable approach for identifying unauthorized data usage in pre-trained diffusion models. To the best of our knowledge, we are the first to discover the performance overestimation of MIAs on diffusion models and present a unified benchmark for more realistic evaluation. Our code is available on GitHub: \url{https://github.com/caradryanl/CopyMark}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。