发现大模型在自然对话中会复现训练数据，最高达100%完全照搬网络内容。
Measuring Non-Adversarial Reproduction of Training Data in Large Language Models
- 通过自然提示测试模型输出与网络文本的重合度。
- 部分模型生成内容高达15%与互联网原文重合，最坏情况达100%。
- 人类写作重合度低，需更强防御机制应对潜在泄露。
大型语言模型会记忆其训练数据的部分内容。记住短语和事实有助于回答世界问题并实现语言流畅性。但已有研究显示，当受到恶意攻击者诱导时，模型可能复现长段落的原始文本。本文研究一种中间状态——非对抗性复现，即在自然、无害提示下，量化模型输出与预训练数据的重合程度。针对多种无害提示类别（如写信或教程），我们发现主流对话模型生成内容中，高达15%与互联网片段重合；最坏情况下，生成内容100%可在线找到原句。相同任务下，人类写作与互联网数据重合度显著更低。我们进一步探究提示策略能否缩小模型与人类之间的复现差距。虽然优化提示能平均降低非对抗性复现，但要防范最坏情况下的训练数据泄露，仍需更强大的防护措施——即便在良性交互中亦然。
原文摘要 · Abstract (English)
Large language models memorize parts of their training data. Memorizing short snippets and facts is required to answer questions about the world and to be fluent in any language. But models have also been shown to reproduce long verbatim sequences of memorized text when prompted by a motivated adversary. In this work, we investigate an intermediate regime of memorization that we call non-adversarial reproduction, where we quantify the overlap between model responses and pretraining data when responding to natural and benign prompts. For a variety of innocuous prompt categories (e.g., writing a letter or a tutorial), we show that up to 15% of the text output by popular conversational language models overlaps with snippets from the Internet. In worst cases, we find generations where 100% of the content can be found exactly online. For the same tasks, we find that human-written text has far less overlap with Internet data. We further study whether prompting strategies can close this reproduction gap between models and humans. While appropriate prompting can reduce non-adversarial reproduction on average, we find that mitigating worst-case reproduction of training data requires stronger defenses -- even for benign interactions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。