arXiv:2606.18767cs.CL2026-06

通过修改输出向量抑制大模型记忆内容,更精准有效。

Output Vector Editing for Memorization Mitigation in Large Language Models

论文配图:Output Vector Editing for Memorization Mitigation in Large Language Models
图 1 · 摘自论文原文
  • 直接编辑MLP层输出向量而非激活值,实现对记忆内容的定向干扰。
  • 在OLMo-7B上对6831条记忆序列实现最高87.9%抑制率,优于零化方法2.7倍。
  • 适用于需隐私保护的场景,尤其适合大规模语言模型安全加固。

大语言模型会记忆并复现训练数据中的序列,带来隐私、版权和安全风险。现有神经元级缓解方法将编辑等同于置零神经元激活,但激活仅决定神经元是否触发,输出向量才是写入残差流并以超位置编码多重特征的关键。本文提出输出向量编辑,一种约束优化的权重编辑方法:定位负责记忆延续的一小部分MLP神经元,最小化修改其输出向量,在词汇空间引入干扰项,重新定向其残差流贡献,同时保持激活不变。在4个模型(360M至7B参数)上评估,重点分析开放权重与预训练语料的OLMo-7B,挖掘出6831条记忆序列,实现最高87.9%抑制率。相比同一神经元集的零化对照,抑制率高出2.7倍,证明效果源于输出向量编辑而非定位本身。四种编辑模式覆盖从激进抑制到最小重定向的谱系,集成后覆盖96.5%的记忆序列,推荐单模式配置达81.5%,无灾难性局部失效。进一步发现约14%序列无法仅靠MLP编辑处理;虽整体非注意力驱动,但剔除主要注意力头可恢复60–64%失败案例,尤其对从前缀复制标记的延续恢复更强,表明注意力是互补后备而非主机制。编辑模式排序与成功-局部性权衡在四模型间迁移,成功率随模型规模提升而非家族差异。

原文摘要 · Abstract (English)

Large language models memorize and reproduce sequences from their training data, creating privacy, copyright, and security risks. Existing neuron-level mitigation methods equate editing with zeroing out neuron activations, but the activation only controls whether a neuron engages; the output vector is what writes to the residual stream and, through superposition, encodes multiple features. We propose output vector editing, a constrained-optimization weight edit that locates a small set of MLP neurons responsible for a memorized continuation and minimally modifies their output vectors to introduce a distractor in vocabulary space, redirecting their residual-stream contributions while leaving activations unchanged. Evaluating on four models from 360M to 7B parameters (SmolLM-360M, OLMo-1B, OLMo-7B, Llama2-7B), we center on OLMo-7B (whose open weights and pretraining corpus enable systematic mining) and mine 6831 memorized sequences, achieving up to 87.9% suppression. The 2.7$\times$ gap over zero ablation on the same located neurons shows the suppression comes from the output-vector edit, not localization alone. Four edit modes span a spectrum from aggressive suppression to minimal redirection; in ensemble they cover 96.5% of memorized sequences, while our recommended single-mode configuration reaches 81.5% with no catastrophic locality failures. We further identify a mechanistic boundary at ${\sim}14%$ of sequences unreachable by MLP-only editing; while these failures are not attention-driven overall, ablating the top contributing attention heads recovers 60--64% of them, with stronger recovery on continuations that copy tokens from the prefix, positioning attention as a complementary fallback rather than a primary mechanism. Edit mode ordering and the success-locality trade-off transfer across all four models, with success rates scaling with model size rather than family.

模型记忆输出编辑隐私保护大模型安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。