arXiv:2606.09865cs.LGcs.CR2026-06

用大模型判断合成表格真伪,发现不同生成模型差异显著。

LLM-as-a-Discriminator: When Synthetic Tables Still Look Real

  • 用大模型做判别器,让其判断表格是真实还是合成的。
  • 在成人和普查数据上,不同模型生成的表格被识别率从0%到100%不等。
  • 相比传统方法,大模型判别更灵敏,适合隐私审计场景。

隐私与数据共享常存在矛盾。许多机构使用合成数据降低隐私风险并实现数据共享。对于表格数据,隐私审计仍具挑战性——在许多情况下,人类也难以分辨表格是真实还是合成的。本文提出一种基于大语言模型(LLM)的判别方法:让LLM对每个表格样本进行真实/合成分类。我们测试两种设置:仅表格(C1)和表格+分布元数据(C2)。采用LLaMA(开源模型)和Gemini(参考模型)进行评估。实验使用三种合成模型(CTGAN、TVAE、Gaussian Copula)在两个公开数据集(UCI Adult 和 ACS Census)上生成数据,共收集451个有效试验。结果显示各模型表现差异明显:在Adult数据集上,LLaMA对报告单元的判别率(DRS)为0%,而Gemini对CTGAN和TVAE的DRS达100%;在Census数据集上,LLaMA多数预测为合成,Gemini在C1中保持高位,但在C2中对CTGAN和TVAE的判别率下降。我们还对比了两样本检验(C2ST)和记录链接作为分布基准,并通过2名标注员完成240次人工测试。结果表明,在合理处理模型选择、厂商报告及数据编码的前提下,LLM判别可作为有效的隐私审计信号。代码与实验脚本已公开于https://github.com/SlokomManel/LLM-as-a-Discriminator。

原文摘要 · Abstract (English)

Privacy and data sharing are often in tension. Many organizations use synthetic data to reduce privacy risk and still share useful data. For tabular data, auditing privacy remains hard. In many cases, even humans cannot easily tell if a table is real or synthetic. In this paper, we propose a method based on LLM discrimination. We ask an LLM to classify each table sample as REAL or SYNTHETIC. We test two settings: C1 with table only, and C2 with table plus distributional metadata. We use LLaMA as an open model and Gemini as a reference model. In our experiments, we run three synthesis models, CTGAN, TVAE, and Gaussian Copula, on two public datasets, UCI Adult and ACS Census. We collect 451 valid trials. Our results show clear differences between models. On Adult, LLaMA reaches DRS=0% in reported cells, while Gemini reaches DRS=100% for CTGAN and TVAE. On Census, LLaMA predicts SYNTHETIC for most samples, while Gemini stays high in C1 but drops for CTGAN and TVAE in C2. We also compare with a classifier two-sample test (C2ST) and record linkage as distributional baselines, and with a human pilot of 2 annotators and 240 trials. Our results show that LLM discrimination is a practical privacy audit signal when model choice, per provider reporting, and data encoding are handled with care. For reproducibility, code and experiment scripts are available at https://github.com/SlokomManel/LLM-as-a-Discriminator.

合成数据隐私审计大模型表格生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。