arXiv:2603.11239cs.AI2026-03被引 1

让大模型可随时撤销修改,避免知识遗忘

Reversible Lifelong Model Editing via Semantic Routing-Based LoRA

  • 用语义路由控制独立的LoRA模块,动态激活编辑内容
  • 支持精确撤销特定修改,恢复模型原始行为
  • 无需额外路由网络,实现端到端编辑决策

现实世界中模型需持续更新,但现有方法在长期编辑过程中仍面临语义漂移或知识遗忘问题。为此,我们提出SoLA——一种基于语义路由的LoRA框架,用于终身模型编辑。每个编辑以独立的LoRA模块封装,训练后冻结,并通过语义路由映射到输入,实现基于语义匹配的动态激活。该机制避免了因聚类更新导致的语义漂移,缓解了参数共享引发的灾难性遗忘。更重要的是,只需移除语义路由中的关键项,即可精准撤销特定编辑,恢复模型原始行为。据我们所知,这是首个实现可逆回滚编辑的框架。此外,SoLA将决策过程集成至编辑层,无需辅助路由网络,支持端到端决策。大量实验表明,SoLA能有效学习并保留编辑知识,实现准确、高效且可逆的终身模型编辑。

原文摘要 · Abstract (English)

The dynamic evolution of real-world necessitates model editing within Large Language Models. While existing methods explore modular isolation or parameter-efficient strategies, they still suffer from semantic drift or knowledge forgetting due to continual updating. To address these challenges, we propose SoLA, a Semantic routing-based LoRA framework for lifelong model editing. In SoLA, each edit is encapsulated as an independent LoRA module, which is frozen after training and mapped to input by semantic routing, allowing dynamic activation of LoRA modules via semantic matching. This mechanism avoids semantic drift caused by cluster updating and mitigates catastrophic forgetting from parameter sharing. More importantly, SoLA supports precise revocation of specific edits by removing key from semantic routing, which restores model's original behavior. To our knowledge, this reversible rollback editing capability is the first to be achieved in existing literature. Furthermore, SoLA integrates decision-making process into edited layer, eliminating the need for auxiliary routing networks and enabling end-to-end decision-making process. Extensive experiments demonstrate that SoLA effectively learns and retains edited knowledge, achieving accurate, efficient, and reversible lifelong model editing.

模型编辑LoRA可逆性语义路由

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。