用序列级概率量化大模型训练数据泄露风险，发现现有评估方法低估了真实威胁。
Sequence-Level Leakage Risk of Training Data in Large Language Models
- 提出序列级概率方法，比传统指标更精细地检测数据泄露
- 发现主流评估指标使泄露风险被低估最多达2.14倍
- 小模型或短前缀反而更容易泄露部分数据，颠覆以往认知
本研究使用序列级概率量化大型语言模型（LLM）的训练数据泄露风险。与以往基于整体提取率的基准测试相比，对单个序列计算提取概率能提供更细粒度的信息。我们在Llama和OPT两个预训练模型上开展研究，分别在Common Crawl和The Pile数据集上训练。结果发现：1）当前主流的提取率指标会将随机化LLM中的数据泄露威胁低估最多达2.14倍；2）尽管平均而言大模型和长前缀可提取更多数据，但30.4%-41.5%的序列在较短前缀或小模型下反而更容易被提取；3）与以往观点相反，在top-k和top-p等常用解码策略下的部分泄露，并不比完整泄露更容易。本工作旨在推动未来研究采用该序列级指标。
原文摘要 · Abstract (English)
This work quantifies the risk of training data leakage from LLMs (Large Language Models) using sequence-level probabilities. Computing extraction probabilities for individual sequences provides finer-grained information than has been studied in prior benchmarking work. We re-analyze the effects of decoding schemes, model sizes, prefix lengths, partial sequence leakages, and token positions to uncover new insights that were not possible in previous works due to their choice of metrics. We perform this study on two pre-trained models, Llama and OPT, trained on the Common Crawl and The Pile respectively. We discover that 1) Extraction Rate, the predominant metric used in prior quantification work, underestimates the threat of leakage of training data in randomized LLMs by as much as 2.14X. 2) Although on average, larger models and longer prefixes can extract more data, this is not true for a substantial portion of individual sequences. 30.4-41.5% of our sequences are easier to extract with either shorter prefixes or smaller models. 3) Contrary to previous beliefs, partial leakage in commonly used decoding schemes like top-k and top-p is not easier than leaking verbatim training data. The aim of this work is to encourage the adoption of this metric for future work on quantification of training data extraction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。