arXiv:2607.13541cs.CRcs.LG2026-07

用AI生成数据训练反而加剧真实数据隐私泄露,研究揭示了这一隐藏风险。

When T2I Synthetic Data Backfires: Amplified Privacy Risks in Real-Synthetic Mix Training

论文配图:When T2I Synthetic Data Backfires: Amplified Privacy Risks in Real-Synthetic Mix Training
图 1 · 摘自论文原文
  • 提出理论框架,证明合成数据会让真实样本更易被模型记忆
  • 设计攻击方法RSMixLeak,实测显示真实数据隐私泄露率显著上升
  • 提供可自检的轻量指标,帮助识别高风险数据集

为应对数据稀缺与隐私限制,学术界和工业界普遍采用真实与文本生成图像(T2I)合成数据混合训练(RSMT)。尽管替换敏感真实样本被视为降低隐私暴露的方法,但参与训练的真实样本所面临的风险长期未受关注。本文首次揭示,RSMT会显著放大真实样本的隐私泄露。我们建立理论框架“RSMT记忆增强”,证明合成数据使真实样本在混合特征空间中移向边缘区域,迫使模型更强烈地记忆它们。基于此,提出RSMixLeak系统评估该风险,包含两种变体:非对抗型用于评估因真实与合成数据固有差异带来的泄漏下限;对抗型则模拟控制生成模型或注入伪造数据的攻击者,通过语义绑定或不可察觉像素涂层扩大分布差距,进一步提升真实数据泄露风险,同时提升下游模型性能。据此,我们还提出仅需真实数据即可计算的轻量级泄漏倾向指标,可有效识别不适合进入RSMT的高风险数据集。

原文摘要 · Abstract (English)

To overcome data scarcity and privacy constraints in data collection, it has become standard practice across academia and industry to augment real training data with text-to-image (T2I)-generated synthetic data, a paradigm we term Real-Synthetic Mix-Training (RSMT). While substituting synthetic data for sensitive real samples is widely regarded as a means to mitigate privacy exposure of the substituted data, the risk to the remaining real samples that actively participate in training has remained largely unexamined. This work reveals, for the first time, that RSMT can substantially amplify privacy leakage of these real training samples. We establish a theoretical framework, RSMT Memorization Amplification, proving that incorporating synthetic data displaces real samples toward peripheral regions of the mixed feature space, in turn forcing the model to memorize them more aggressively. Guided by this foundation, we propose RSMixLeak to systematically assess this risk through membership inference attacks (MIAs). RSMixLeak comprises two variants depending on the adversary's capability. The non-adversarial variant audits a benign RSMT pipeline with an honest T2I provider, establishing a lower bound on the leakage induced by the intrinsic gap between real and T2I-generated data. The adversarial variant considers an adversary who controls the T2I model or contributes crafted data to the T2I provider, and deliberately enlarges this distributional gap on a target class via either high-level semantic attribute binding or imperceptible pixel-level coating, further amplifying leakage on real training data while improving downstream model utility. Motivated by these findings, we further propose a lightweight leakage propensity indicator computable from real data alone that reliably identifies high-risk datasets unsuitable for entering RSMT, as a self-assessable mitigation.

隐私泄露生成数据模型记忆安全评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。