解决大模型批量知识编辑中的语义混淆问题，提升准确性。
Orthogonal Representation Editing: Decoupling Semantic Entanglement in Batch Knowledge Editing of LLMs
- 通过正交约束解耦语义纠缠，优化隐藏层表示空间的编辑
- 在多语言场景下显著优于现有方法，保持高编辑精度
- 适合需要精准、批量更新知识的大模型应用
知识编辑旨在不进行完整重训练的情况下高效更新大语言模型（LLMs）中的事实信息。然而，现有方法在批量知识编辑中仍存在性能下降问题。我们发现，语义表示纠缠（如概念重叠和共享句法模式）会在表示空间中累积干扰，降低编辑精度。为此，本文提出正交表示编辑（ORE），在LLM的隐藏表示空间中进行编辑，构建通用语义子空间，并对编辑向量施加正交约束，有效解耦语义纠缠。此外，引入门控非线性表示头，实现编辑位置的自适应学习和知识注入的精确控制。大量实验表明，ORE优于现有方法，在跨语言知识编辑场景中表现更优。代码已开源：https://github.com/YVVH/ORE。
原文摘要 · Abstract (English)
Knowledge editing aims to efficiently update factual information in Large Language Models (LLMs) without full retraining. However, existing methods still suffer from performance degradation in batch knowledge editing. We identify that semantic representation entanglement, such as overlapping concepts and shared syntactic patterns, accumulates interference in the representation space and reduces editing precision. To bridge this gap, in this paper, we propose Orthogonal Representation Editing (ORE), which performs edits in the hidden representation space of LLMs by constructing a general semantic subspace and enforcing orthogonal constraints on edit vectors, effectively decoupling semantic entanglement. Furthermore, we introduce a gated non-linear representation head to enable adaptive learning of editing locations and precise control over knowledge injection. Extensive experiments show that ORE outperforms existing methods and achieves superior performance in cross-lingual knowledge editing scenarios. We release our code at https://github.com/YVVH/ORE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。