让大模型知识编辑跨模型通用，一次编辑长期有效
MindBridge: Scalable and Cross-Model Knowledge Editing via Memory-Augmented Modality
- 用独立记忆模态存储编辑内容，不依赖具体模型
- 支持数万条知识批量编辑，跨模型效果稳定
- 适合需要频繁更新知识的开源大模型使用者
知识编辑是一种高效准确更新大语言模型（LLMs）知识以缓解过时和纠正错误的技术。然而，现有方法大多针对特定模型设计，导致每次LLM更新后编辑内容被丢弃，需反复重编辑，尤其在快速演进的开源社区中负担沉重。为此，我们提出跨模型知识编辑问题，并引入MindBridge，该方案受多模态模型中模态处理低耦合启发。MindBridge提出记忆模态新概念，将编辑知识编码为独立模态，先进行与LLM无关的预训练，再灵活集成到各类LLM中。在多个LLM和主流知识编辑数据集上的大量实验表明，即便面对数万条知识编辑，MindBridge仍表现优异，且能灵活适配不同模型。代码已开源：https://github.com/CrashBugger/MindBridge。
原文摘要 · Abstract (English)
Knowledge editing is a technique for efficiently and accurately updating the knowledge of large language models (LLMs) to alleviate obsolescence and correct errors. However, most existing methods overfit to specific models, causing edited knowledge to be discarded during each LLM update and requiring frequent re-editing, which is particularly burdensome in today's rapidly evolving open-source community. To address this issue, we propose the problem of cross-model knowledge editing and introduce MindBridge, a scalable solution inspired by the low coupling between modality processing and LLMs in multi-modal models. MindBridge introduces the novel concept of memory modality, which encodes edited knowledge as an independent modality. It first performs LLM-agnostic pre-training of the memory modality and then integrates it with various LLMs. Extensive experiments on multiple LLMs and popular knowledge editing datasets demonstrate that MindBridge achieves superior performance even in editing tens of thousands of knowledge entries and can flexibly adapt to different LLMs. Our code is available at https://github.com/CrashBugger/MindBridge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。