arXiv:2505.22322cs.LG2025-05被引 3

发现表格扩散模型中少数样本泄露严重,提出可迁移的动态剪枝方法

A Closer Look on Memorization in Tabular Diffusion Model: A Data-Centric Perspective

  • 按生成样本相似度量化每条数据的遗忘程度,识别高泄露样本
  • 仅移除约10%的高记忆样本即可大幅降低泄露,且不影响数据多样性
  • 方法不依赖模型结构,可跨扩散模型、GAN、VAE通用

扩散模型在生成高质量表格数据方面表现优异,但存在复制训练样本的隐私风险。现有工作多从数据集层面增强来缓解,却未揭示哪些具体样本导致泄漏。本文首次从数据视角研究表格扩散模型的记忆动态。通过相对距离比衡量每条真实样本的生成复制品数量,量化其记忆程度。实证分析显示记忆分布呈重尾特征:极少数样本贡献了大部分泄露,经样本删除实验验证。进一步将样本分为高/低记忆组,追踪其训练过程中的首次记忆时间与每轮记忆强度(AUC)。结果显示高记忆样本在早期训练中更早被记住且信号更强。基于此,提出两阶段、模型无关的缓解方法DynamicCut:(a) 按epoch级强度排序样本,(b) 剪除可调比例的前缀,(c) 用过滤后数据重新训练。在多个表格数据集和模型上,DynamicCut显著降低记忆现象,对数据多样性和下游性能影响小,并可与增强类防御互补。此外,该方法具备跨模型迁移性:在一个模型中识别出的高记忆样本,移除后也能有效降低其他模型(如GAN、VAE)的记忆水平。

原文摘要 · Abstract (English)

Diffusion models have shown strong performance in generating high-quality tabular data, but they carry privacy risks by reproducing exact training samples. While prior work focuses on dataset-level augmentation to reduce memorization, little is known about which individual samples contribute most. We present the first data-centric study of memorization dynamics in tabular diffusion models. We quantify memorization for each real sample based on how many generated samples are flagged as replicas, using a relative distance ratio. Our empirical analysis reveals a heavy-tailed distribution of memorization counts: a small subset of samples contributes disproportionately to leakage, confirmed via sample-removal experiments. To understand this, we divide real samples into top- and non-top-memorized groups and analyze their training-time behaviors. We track when each sample is first memorized and monitor per-epoch memorization intensity (AUC). Memorized samples are memorized slightly earlier and show stronger signals in early training. Based on these insights, we propose DynamicCut, a two-stage, model-agnostic mitigation method: (a) rank samples by epoch-wise intensity, (b) prune a tunable top fraction, and (c) retrain on the filtered dataset. Across multiple tabular datasets and models, DynamicCut reduces memorization with minimal impact on data diversity and downstream performance. It also complements augmentation-based defenses. Furthermore, DynamicCut enables cross-model transferability: high-ranked samples identified from one model (e.g., a diffusion model) are also effective for reducing memorization when removed from others, such as GANs and VAEs.

表格生成隐私保护扩散模型记忆分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。