让大模型任意编辑诗歌代码等长文本知识，效果比现有方法提升21.5%
AnyEdit: Edit Any Knowledge Encoded in Language Models
- 将长文本分块逐段编辑关键词，实现多格式知识精准更新
- 在多个基准上相比强基线提升21.5%准确率，新数据集验证效果
- 可插件式集成现有方法，支持任意长度和格式的知识修改
大型语言模型常生成错误或过时信息，亟需高效精准的知识更新。现有编辑方法难以处理诗歌、代码片段、数学推导等多样格式的长文本知识，根源在于其仅依赖单个标记隐藏状态的修改，我们称之为“效能瓶颈”。为此，我们提出AnyEdit，一种新的自回归编辑范式。它将长文本知识分解为顺序块，逐块迭代编辑关键标记，确保输出一致准确。理论上，我们基于互信息链式法则证明了AnyEdit可更新任意知识。实证上，它在UnKEBench、AKEW及我们新构建的EditEverything数据集上的表现优于强基线21.5%。此外，AnyEdit作为即插即用框架，使现有编辑方法能处理任意长度与格式的知识，显著拓展了大模型知识编辑的应用范围与实用性。
原文摘要 · Abstract (English)
Large language models (LLMs) often produce incorrect or outdated information, necessitating efficient and precise knowledge updates. Current model editing methods, however, struggle with long-form knowledge in diverse formats, such as poetry, code snippets, and mathematical derivations. These limitations arise from their reliance on editing a single token's hidden state, a limitation we term "efficacy barrier". To solve this, we propose AnyEdit, a new autoregressive editing paradigm. It decomposes long-form knowledge into sequential chunks and iteratively edits the key token in each chunk, ensuring consistent and accurate outputs. Theoretically, we ground AnyEdit in the Chain Rule of Mutual Information, showing its ability to update any knowledge within LLMs. Empirically, it outperforms strong baselines by 21.5% on benchmarks including UnKEBench, AKEW, and our new EditEverything dataset for long-form diverse-formatted knowledge. Additionally, AnyEdit serves as a plug-and-play framework, enabling current editing methods to update knowledge with arbitrary length and format, significantly advancing the scope and practicality of LLM knowledge editing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。