用大模型生成真实表格数据，关键提升特征与标签关联性。
Generating Realistic Tabular Data with Large Language Models
- 输入数据采用新排列策略微调模型
- 通过特征条件采样生成样本，再用提示词生成标签
- 生成数据让分类器表现接近真实数据，适合数据隐私场景
尽管生成模型在图像生成方面成果显著，但针对表格数据生成的研究较少。近期，大语言模型（LLM）在多种任务中表现优异，也被用于表格数据生成。然而，现有方法未能准确捕捉特征与目标变量之间的相关性，限制了其在下游预测任务中的应用。为此，我们提出一种基于LLM的方法，包含三项关键改进：第一，在微调阶段采用新型数据排列策略；第二，提出特征条件采样生成合成样本；第三，基于生成样本构建提示词，查询微调后的LLM以生成标签。大量实验表明，该方法在20个数据集上显著优于10个SOTA基线，在下游任务中表现优异。生成的样本在质量与多样性方面均高度真实。更重要的是，使用合成数据训练的分类器在半数基准数据集上可媲美原始数据训练的模型，是表格数据生成领域的重大突破。
原文摘要 · Abstract (English)
While most generative models show achievements in image data generation, few are developed for tabular data generation. Recently, due to success of large language models (LLM) in diverse tasks, they have also been used for tabular data generation. However, these methods do not capture the correct correlation between the features and the target variable, hindering their applications in downstream predictive tasks. To address this problem, we propose a LLM-based method with three important improvements to correctly capture the ground-truth feature-class correlation in the real data. First, we propose a novel permutation strategy for the input data in the fine-tuning phase. Second, we propose a feature-conditional sampling approach to generate synthetic samples. Finally, we generate the labels by constructing prompts based on the generated samples to query our fine-tuned LLM. Our extensive experiments show that our method significantly outperforms 10 SOTA baselines on 20 datasets in downstream tasks. It also produces highly realistic synthetic samples in terms of quality and diversity. More importantly, classifiers trained with our synthetic data can even compete with classifiers trained with the original data on half of the benchmark datasets, which is a significant achievement in tabular data generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。