通过输入扰动分析，检测大模型是否在复述训练数据而非真正理解。
Memorization or Interpolation ? Detecting LLM Memorization through Input Perturbation Analysis
- 基于输入扰动观察输出一致性变化，判断模型是记忆还是泛化。
- 在Pythia和GPT-4o上验证，能识别圣经、代码等已知数据的复现。
- 无需访问模型内部，适合评估隐私与版权风险，适合安全团队使用。
大型语言模型（LLMs）虽在海量数据上训练取得优异性能，但存在直接复现训练数据的问题，引发数据隐私、知识产权及评估可靠性担忧。本文提出PEARL方法，通过分析输入扰动对模型输出一致性的影响，实现不依赖模型内部信息的内存检测。在Pythia开源模型上开展广泛实验，构建了可靠的识别框架，可判断模型是否仅重复已有信息。应用于GPT-4o模型时，成功识别出对《圣经》经典文本及HumanEval常见代码的复现，并提供证据表明纽约时报新闻文章可能已被纳入训练数据。
原文摘要 · Abstract (English)
While Large Language Models (LLMs) achieve remarkable performance through training on massive datasets, they can exhibit concerning behaviors such as verbatim reproduction of training data rather than true generalization. This memorization phenomenon raises significant concerns about data privacy, intellectual property rights, and the reliability of model evaluations. This paper introduces PEARL, a novel approach for detecting memorization in LLMs. PEARL assesses how sensitive an LLM's performance is to input perturbations, enabling memorization detection without requiring access to the model's internals. We investigate how input perturbations affect the consistency of outputs, enabling us to distinguish between true generalization and memorization. Our findings, following extensive experiments on the Pythia open model, provide a robust framework for identifying when the model simply regurgitates learned information. Applied on the GPT 4o models, the PEARL framework not only identified cases of memorization of classic texts from the Bible or common code from HumanEval but also demonstrated that it can provide supporting evidence that some data, such as from the New York Times news articles, were likely part of the training data of a given model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。