模型能泄露训练数据,但平时并不愿意,需评估真实泄漏风险。
LLMs Can Leak Training Data But Do They Want To? A Propensity-Aware Evaluation of Memorization in LLMs

- 提出新评估框架PropMe,区分恶意攻击与正常使用下的记忆行为。
- 发现模型在诱导下记忆强烈,但日常使用中泄漏倾向很低。
- 建议审计时同时报告最坏情况和普通场景的泄漏概率。
大型语言模型可能复现训练数据,但现有评估多聚焦于是否能被强制泄露,而非在常规使用中是否会泄露。本文提出PropMe框架,通过对比基于前缀的能力攻击与非对抗性评估,实现对记忆倾向的量化。引入指标转换方法,可将现有函数转化为倾向性度量。进一步提出SimpleTrace,一个基于infinigram的轻量级追踪流程,可确定性地将模型输出归因于大规模训练语料库,并计算原文、近似原文及倾向性转换后的记忆度量。在两个全开源模型(Comma与DFM Decoder)上,分别在Common Pile与Dynaword数据集上进行双语测试,结果表明:能力与倾向之间存在显著差距——前缀攻击引发强记忆信号,而通用或数据集特定提示下的倾向性得分普遍偏低。即模型在被直接诱导时可暴露训练数据,但在常见非对抗性场景中极少发生。此外,持续预训练的DFM Decoder相比Comma在Common Pile上的记忆能力和倾向性均下降,验证了后续训练数据差异可降低记忆风险。研究建议记忆审计应同时报告最坏情况可提取性与日常泄漏倾向,以更全面理解该现象。
原文摘要 · Abstract (English)
Large language models can reproduce training data, but existing memorization evaluations mostly measure whether models can be forced to do so, rather than whether they do so under ordinary use. We introduce PropMe, a propensity-aware framework for memorization evaluation that contrasts prefix-based capability attacks with non-adversarial evaluations. We propose a metric transformation that, applied to existing functions, allows to create propensity metrics. We further introduce SimpleTrace, a lightweight tracing pipeline built on infini-gram that deterministically attributes model generations to large-scale training corpora and computes verbatim, near-verbatim, and propensity-transformed memorization metrics. Evaluating two fully-open models: Comma and DFM Decoder on two datasets: Common Pile and Dynaword in two languages, we find a consistent gap between capability and propensity: prefix attacks elicit substantially stronger memorization signals than generic or dataset-specific prompts, while propensity scores remain low overall. Thus, the models can reveal training data when directly elicited, but rarely do so in more common non-adversarial settings. We also find that DFM Decoder, which is continually pre-trained from Comma, exhibits reduced memorization and memorization propensity for Common Pile, confirming that memorization capability can decrease when later training emphasizes partially different data. Our results suggest, and we encourage, that memorization audits should report both worst-case extractability and ordinary leakage propensity in order to have a more comprehensive view of this phenomenon.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。