通过正交投影避免多语言知识编辑时的参数干扰
Mitigating Negative Interference in Multilingual Sequential Knowledge Editing through Null-Space Constraints
- 将每种语言的更新投影到此前更新空间的正交补空间
- 在六种语言上验证,显著降低参数干扰并提升准确性
- 适合需要高效多语言知识维护的系统开发者
在大型语言模型中高效更新多语言知识,同时保持跨语言事实表征的一致性,仍是长期未解难题。尽管为每种语言部署独立编辑系统看似可行,但需管理多个模型,成本高昂。更优方案是将所有语言的知识更新整合至统一模型。然而,跨语言顺序编辑常引发破坏性参数干扰,严重削弱多语言泛化能力与注入知识的准确性。为此,我们提出LangEdit,一种基于零空间约束的新框架,可精确隔离各语言特定的知识更新。其核心创新在于将每种语言的参数更新投影至先前更新子空间的正交补空间,数学上保证更新独立性,同时保留多语言泛化能力。我们在三种模型架构、六种语言及四项下游任务上进行综合评估,结果表明LangEdit有效缓解参数干扰,优于现有最先进编辑方法。实验验证其在大模型中实现高效且准确的多语言知识更新的潜力。代码已公开于https://github.com/VRCMF/LangEdit.git。
原文摘要 · Abstract (English)
Efficiently updating multilingual knowledge in large language models (LLMs), while preserving consistent factual representations across languages, remains a long-standing and unresolved challenge. While deploying separate editing systems for each language might seem viable, this approach incurs substantial costs due to the need to manage multiple models. A more efficient solution involves integrating knowledge updates across all languages into a unified model. However, performing sequential edits across languages often leads to destructive parameter interference, significantly degrading multilingual generalization and the accuracy of injected knowledge. To address this challenge, we propose LangEdit, a novel null-space constrained framework designed to precisely isolate language-specific knowledge updates. The core innovation of LangEdit lies in its ability to project parameter updates for each language onto the orthogonal complement of previous updated subspaces. This approach mathematically guarantees update independence while preserving multilingual generalization capabilities. We conduct a comprehensive evaluation across three model architectures, six languages, and four downstream tasks, demonstrating that LangEdit effectively mitigates parameter interference and outperforms existing state-of-the-art editing methods. Our results highlight its potential for enabling efficient and accurate multilingual knowledge updates in LLMs. The code is available at https://github.com/VRCMF/LangEdit.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。