arXiv:2412.11044cs.LG2024-12ICML被引 9

发现表格扩散模型会记忆训练数据,提出新方法有效缓解此问题。

Understanding and Mitigating Memorization in Diffusion Models for Tabular Data

  • 通过特征片段交换实现数据增强,防止模型复制原始数据。
  • 实验显示该方法在多个数据集上显著降低记忆现象。
  • 适合关注数据隐私与生成质量的研究者使用。

近年来,表格数据生成受到广泛关注,表格扩散模型大幅提升了合成数据质量。然而,尽管图像和文本生成中的记忆现象已深入研究,其在表格数据中的影响仍不明确。本文首次系统探究了表格扩散模型中的记忆问题。实证分析表明,记忆现象确实存在,且随训练轮次增加而加剧。我们进一步考察了数据集大小、特征维度及不同扩散模型对记忆的影响,并提供了理论解释。为此,提出TabCutMix:一种简单有效的数据增强方法,随机交换同类别样本的特征片段。在此基础上,提出改进版TabCutMixPlus,基于特征相关性聚类,确保同一簇内特征整体交换,保持特征一致性,避免分布外生成。在多个数据集与扩散模型上的实验表明,该方法能有效缓解记忆问题,同时维持高质量数据生成。

原文摘要 · Abstract (English)

Tabular data generation has attracted significant research interest in recent years, with the tabular diffusion models greatly improving the quality of synthetic data. However, while memorization, where models inadvertently replicate exact or near-identical training data, has been thoroughly investigated in image and text generation, its effects on tabular data remain largely unexplored. In this paper, we conduct the first comprehensive investigation of memorization phenomena in diffusion models for tabular data. Our empirical analysis reveals that memorization appears in tabular diffusion models and increases with larger training epochs. We further examine the influence of factors such as dataset sizes, feature dimensions, and different diffusion models on memorization. Additionally, we provide a theoretical explanation for why memorization occurs in tabular diffusion models. To address this issue, we propose TabCutMix, a simple yet effective data augmentation technique that exchanges randomly selected feature segments between random same-class training sample pairs. Building upon this, we introduce TabCutMixPlus, an enhanced method that clusters features based on feature correlations and ensures that features within the same cluster are exchanged together during augmentation. This clustering mechanism mitigates out-of-distribution (OOD) generation issues by maintaining feature coherence. Experimental results across various datasets and diffusion models demonstrate that TabCutMix effectively mitigates memorization while maintaining high-quality data generation.

扩散模型表格生成数据隐私数据增强

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。