arXiv:2512.06062cs.LGcs.AI2025-12

生成式模型合成数据仍会泄露原始数据信息,攻击者可借此推断成员身份。

When Privacy Isn't Synthetic: Hidden Data Leakage in Generative AI Models

  • 通过大量生成样本聚类,定位真实数据密集区域作为攻击线索。
  • 在医疗、金融等场景中,合成数据与真实数据存在结构重叠导致隐私泄露。
  • 无需模型内部或真实数据,黑盒攻击即可实现成员推理,适合安全评估者参考。

生成式模型常被用于生成隐私保护的合成数据,作为敏感训练数据的安全替代方案。然而,我们证明此类合成数据仍可能通过数据流形的结构重叠泄露原始训练样本信息。本文提出一种黑盒成员推理攻击:攻击者反复查询生成模型获取大量合成样本,采用无监督聚类识别合成分布中的高密度区域,并分析其簇中心及邻域,这些区域对应原始训练数据的高密度区,可作为训练样本的代理,使攻击者推断成员身份或重建近似记录。在医疗、金融等敏感领域实验表明,即使生成器使用差分隐私或其他噪声机制,真实数据与合成数据的簇重叠仍会导致可观测的成员泄漏。结果揭示了合成数据生成流程中一个未充分关注的攻击面,呼吁更强的隐私保障需考虑分布邻域推断,而不仅是样本级记忆。代码已公开于 github.com/Cluster-Medoid-Leakage-Attack。

原文摘要 · Abstract (English)

Generative models are increasingly used to produce privacy-preserving synthetic data as a safe alternative to sharing sensitive training datasets. However, we demonstrate that such synthetic releases can still leak information about the underlying training samples through structural overlap in the data manifold. We propose a black-box membership inference attack that exploits this vulnerability without requiring access to model internals or real data. The attacker repeatedly queries the generative model to obtain large numbers of synthetic samples, performs unsupervised clustering to identify dense regions of the synthetic distribution, and then analyzes cluster medoids and neighborhoods that correspond to high-density regions in the original training data. These neighborhoods act as proxies for training samples, enabling the adversary to infer membership or reconstruct approximate records. Our experiments across healthcare, finance, and other sensitive domains show that cluster overlap between real and synthetic data leads to measurable membership leakage-even when the generator is trained with differential privacy or other noise mechanisms. The results highlight an under-explored attack surface in synthetic data generation pipelines and call for stronger privacy guarantees that account for distributional neighborhood inference rather than sample-level memorization alone, underscoring its role in privacy-preserving data publishing. Implementation and evaluation code are publicly available at:github.com/Cluster-Medoid-Leakage-Attack.

生成模型隐私泄露成员推理合成数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。