一次编辑,多模型同步更新知识,解决大模型过时问题
One for All: Update Parameterized Knowledge Across Multiple Models
- 用插件式模块统一更新多个大模型参数
- 在不同模型上实现稳定且高效的精准知识修改
- 适合需要多模型协同更新知识的工程场景
大型语言模型虽蕴含丰富世界知识,但难以保持最新状态,常导致错误和幻觉。知识编辑提供了一种无需重新训练的高效替代方案,可通过修改特定参数实现定向更新。然而,现有方法多针对单个模型,难以高效更新多个模型或适配新模型。为此,我们提出OnceEdit,一种基于模型集成的新型方法,采用插件式编辑模块,在多个模型间实现稳定的知识更新。基于模型集成,OnceEdit引入两项关键机制:首先,通过 extit{weight}标记动态分配权重,区分需编辑与非编辑样本,确保整合模型知识的有效利用;其次,引入集成增强机制,缓解模型集成中对中心模型的过度依赖,提升知识编辑适用性。在多种大模型上的大量实验表明,OnceEdit持续优于现有方法,并实现更优的编辑效率。进一步分析验证了其在多模型编辑场景下的适应性与稳定性。代码将公开。
原文摘要 · Abstract (English)
Large language models (LLMs) encode vast world knowledge but struggle to stay up-to-date, often leading to errors and hallucinations. Knowledge editing offers an efficient alternative to retraining, enabling targeted modifications by updating specific model parameters. However, existing methods primarily focus on individual models, posing challenges in efficiently updating multiple models and adapting to new models. To address this, we propose OnceEdit, a novel ensemble-based approach that employs a plug-in model as the editing module, enabling stable knowledge updates across multiple models. Building on the model ensemble, OnceEdit introduces two key mechanisms to enhance its effectiveness. First, we introduce a dynamic weight mechanism through a \weight token for distinguishing between edit-related and non-edit-related instances, ensuring the appropriate utilization of knowledge from integrated models. Second, we incorporate an ensemble enhancement mechanism to mitigate the excessive reliance on the central model inherent in the model ensemble technique, making it more suitable for knowledge editing. Extensive experiments on diverse LLMs demonstrate that OnceEdit consistently outperforms existing methods while achieving superior editing efficiency. Further analysis confirms its adaptability and stability in multi-model editing scenarios. Our code will be available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。