发现大模型训练数据有可被识别的指纹，且会随训练传播。
Measuring Fingerprints of Web-filtered Text Datasets and Fingerprint Propagation Through Training
- 通过分类实验发现文本数据集存在可区分指纹。
- 模型生成内容仍能被准确识别来源数据集。
- 适合关注数据隐私与模型溯源的研究者阅读。
我们通过数据集分类实验研究了大语言模型预训练数据中的指纹问题。分析了基于CommonCrawl构建的多个开源数据集，包括C4、RefinedWeb、DolmaCC、RedPajama-V2、FineWeb和DCLM-Baseline。尽管这些数据集经过相似的清洗流程，神经网络仍能以远超人类的准确率判断单个文本序列所属的数据集，表明过滤与处理流程的微小差异会引入指纹。这些指纹体现在格式、词汇和内容分布上，可能损害跨数据集泛化能力。此外，我们发现这些指纹会传播至训练过程：由这些数据集训练出的模型生成的内容，仍能被基于原始数据集训练的分类器准确识别。这为揭示通常不公开的数据特征提供了线索，例如预训练混合比例和微调数据来源。
原文摘要 · Abstract (English)
We investigate fingerprints in pretraining datasets for large language models (LLMs) through dataset classification experiments. Building on prior work demonstrating the existence of fingerprints or biases in popular computer vision datasets, we analyze popular open-source pretraining datasets for LLMs derived from CommonCrawl including C4, RefinedWeb, DolmaCC, RedPajama-V2, FineWeb, and DCLM-Baseline. Despite those datasets being obtained with similar curation steps, neural networks can classify surprisingly well which dataset a single text sequence belongs to, significantly better than a human can. This indicates that small differences in filtering and processing pipelines induce fingerprints. Those fingerprints are evident in formatting, vocabulary, and content distributions, and can negatively impact cross-dataset generalization. Additionally, we show that these fingerprints propagate through training: sequences generated by models trained on those datasets can be accurately classified by a classifier trained on the original datasets. This can offer insights into data characteristics that are typically undisclosed by LLM developers, including pretraining mixture proportions and finetuning data sources.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。