高内在维度序列更难被语言模型记忆，尤其在大模型和稀疏训练下。
Memorization in Language Models through the Lens of Intrinsic Dimension
- 用内在维度衡量序列在隐空间的结构复杂度，分析其对记忆的影响。
- 高内在维度序列的记忆概率显著降低，尤其在参数量大、暴露少时。
- 为理解大模型隐私风险提供新视角，适合关注模型安全的研究者。
语言模型在训练过程中容易无意中记忆数据片段，并在生成时泄露，引发隐私与知识产权担忧。尽管已有研究指出上下文长度、参数量和重复频率是关键影响因素，但对潜在结构如何调控记忆率仍不清楚。本文探究了内在维度（ID）——一种表征序列在隐空间结构复杂度的几何指标——在记忆过程中的作用。结果表明，内在维度对记忆具有抑制作用：相比低内在维度序列，高内在维度序列更难被记忆，尤其在过参数化模型和稀疏暴露条件下。该发现揭示了规模、暴露程度与结构复杂度三者交互对记忆行为的影响。
原文摘要 · Abstract (English)
Language Models (LMs) are prone to memorizing parts of their data during training and unintentionally emitting them at generation time, raising concerns about privacy leakage and disclosure of intellectual property. While previous research has identified properties such as context length, parameter size, and duplication frequency, as key drivers of unintended memorization, little is known about how the latent structure modulates this rate of memorization. We investigate the role of Intrinsic Dimension (ID), a geometric proxy for the structural complexity of a sequence in latent space, in modulating memorization. Our findings suggest that ID acts as a suppressive signal for memorization: compared to low-ID sequences, high-ID sequences are less likely to be memorized, particularly in overparameterized models and under sparse exposure. These findings highlight the interaction between scale, exposure, and complexity in shaping memorization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。