揭示冗余度是影响大模型记忆隐私的关键因素。
Beyond Frequency: The Role of Redundancy in Large Language Model Memorization
- 通过扰动前缀分析发现，低冗余样本更易被记忆
- 79%的记忆样本为低冗余，其脆弱性是高冗余的2倍
- 高冗余内容既更易记忆也更易被扰动破坏
大语言模型中的记忆现象对隐私和公平性构成重大风险。尽管已有研究指出词频和重复模式与记忆相关，但本研究发现：词频提升对记忆样本的影响极小（如0.09），却显著影响非记忆样本（如0.25），且该趋势在不同模型规模下一致。通过扰动样本前缀并量化扰动强度（基于词元位置变化），我们发现冗余度与记忆模式密切相关。结果表明：约79%的记忆样本为低冗余，其脆弱性是高冗余样本的2倍；记忆样本在扰动下下降0.6，而非记忆样本仅下降0.01，说明冗余内容虽更易被记住，但也更脆弱。该发现提示可通过冗余引导的数据预处理降低隐私风险，缓解偏差，保障模型部署公平性。
原文摘要 · Abstract (English)
Memorization in large language models poses critical risks for privacy and fairness as these systems scale to billions of parameters. While previous studies established correlations between memorization and factors like token frequency and repetition patterns, we revealed distinct response patterns: frequency increases minimally impact memorized samples (e.g. 0.09) while substantially affecting non-memorized samples (e.g., 0.25), with consistency observed across model scales. Through counterfactual analysis by perturbing sample prefixes and quantifying perturbation strength through token positional changes, we demonstrate that redundancy correlates with memorization patterns. Our findings establish that: about 79% of memorized samples are low-redundancy, these low-redundancy samples exhibit 2-fold higher vulnerability than high-redundancy ones, and consequently memorized samples drop by 0.6 under perturbation while non-memorized samples drop by only 0.01, indicating that more redundant content becomes both more memorable and more fragile. These findings suggest potential redundancy-guided approaches for data preprocessing, thereby reducing privacy risks and mitigating bias to ensure fairness in model deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。