让大模型更新知识时不破坏原有知识，靠智能路由专家模块实现。
UniAdapt: A Universal Adapter for Knowledge Calibration
- 用向量路由机制动态选择知识专家，避免冲突
- 在多个评测中优于现有长期编辑方法，效果显著
- 无需修改原模型，适合各类大语言模型快速集成
大型语言模型需频繁更新以纠正错误并跟上持续演进的知识。近期模型编辑研究揭示了在终身学习场景下保持泛化性与局部性平衡的挑战。我们发现直接向模型注入知识常引发冲突，并可能破坏其他无关预训练知识。为此，提出UniAdapt——一种通用知识校准适配器。受专家混合架构与检索增强生成启发，UniAdapt采用向量辅助路由机制，通过语义相似度搜索构建路由向量，动态将输入分发至相应专家。该设计完全模型无关，支持即插即用。实验表明，UniAdapt优于现有终身模型编辑方法，在多数指标上表现优异。
原文摘要 · Abstract (English)
Large Language Models (LLMs) require frequent updates to correct errors and keep pace with continuously evolving knowledge in a timely and effective manner. Recent research in it model editing has highlighted the challenges in balancing generalization and locality, especially in the context of lifelong model editing. We discover that inserting knowledge directly into the model often causes conflicts and potentially disrupts other unrelated pre-trained knowledge. To address this problem, we introduce UniAdapt, a universal adapter for knowledge calibration. Inspired by the Mixture of Experts architecture and Retrieval-Augmented Generation, UniAdapt is designed with a vector-assisted router that is responsible for routing inputs to appropriate experts. The router maintains a vector store, including multiple shards, to construct routing vectors based on semantic similarity search results. UniAdapt is fully model-agnostic and designed for seamless plug-and-play integration. Experimental results show that UniAdapt outperforms existing lifelong model editors and achieves exceptional results in most metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。