用哈希记忆库实现大规模序列化模型编辑，稳定支持超5万次更新。
HoReN: Normalized Hopfield Retrieval for Large-Scale Sequential Model Editing
- 基于离散键值内存和霍普菲尔德动态检索，通过角度相似性定位知识
- 在ZsRE上支持5万次连续编辑，整体性能保持0.93以上，优于旧方法
- 适合需长期维护的大型语言模型，尤其擅长处理改写句式仍准确
大语言模型部署后常出现事实过时或错误，但重训练成本过高。为此提出霍普菲尔德归一化编辑器HoReN，一种基于代码本的参数保留型编辑方法。它在单个MLP层外嵌入离散键值记忆库，将每个代码本条目同时视为知识键与霍普菲尔德存储模式。通过单位超球面的角度相似性检索编辑内容，并利用阻尼霍普菲尔德动力学优化查询，使同义改写句收敛至正确记忆域，无关输入保持稳定。HoReN在涵盖标准ZsRE、结构化WikiBigEdit及非结构化UnKE的多个基准上表现优异。特别地，在ZsRE上可实现5万次连续编辑，整体性能稳定高于0.93，而以往编辑器在未达1万次时已严重退化。代码已开源。
原文摘要 · Abstract (English)
Large language models encode vast factual knowledge that can become outdated or incorrect after deployment, yet retraining is prohibitively costly. This motivates lifelong model editing, which updates targeted behavior while preserving the rest of the model. Existing editors, both parameter-modifying and parameter-preserving, degrade severely as edits accumulate and struggle to generalize across paraphrases. We propose HoReN, a codebook-based parameter-preserving editor that wraps a single MLP layer with a discrete key-value memory. HoReN treats each codebook entry as both a knowledge key and a Hopfield stored pattern, retrieves edits by angular similarity on the unit hypersphere, and refines queries through damped Hopfield dynamics so paraphrases converge to the correct memory basin while unrelated inputs remain stable. HoReN achieves strong editing performance with consistent gains across diverse benchmarks spanning standard ZsRE, structured WikiBigEdit, and unstructured UnKE evaluations. Moreover, HoReN scales to 50K sequential edits on ZsRE with stable overall performance above 0.93, while prior editors collapse or degrade severely before reaching 10K. Our code is available at https://github.com/ha11ucin8/HoReN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。