arXiv:2510.10398cs.CLcs.AI2025-10EMNLP被引 2

让大模型更新知识时更连贯,避免碎片化记忆。

STEAM: A Semantic-Level Knowledge Editing Framework for Large Language Models

  • 通过语义锚点引导新知识融入模型内部表示
  • 编辑后推理能力提升,语义一致性显著改善
  • 适合需要可靠知识更新的场景,如医疗问答

大语言模型在预训练阶段获取了大量事实知识,但这些知识是静态的,仅反映训练时的世界状态。知识编辑成为无需全量重训即可更新过时或错误信息的可行方案。然而,现有定位-编辑方法多聚焦于词元级概率优化,忽视语义连贯性。分析发现,此类编辑后的知识常以孤立残差流形式存在于模型隐空间,与原有知识分离,绕过自然推理过程。为此,我们提出 extsc{Steam}:一种语义层面的知识编辑框架,通过识别目标语义锚点,并在优化过程中引入对齐损失,引导编辑后的事实表示向这些锚点靠拢,从而实现新知识与模型原有知识结构的融合。实验表明, extsc{Steam} 显著提升了模型对编辑知识的推理能力与语义一致性,凸显了隐空间对齐在可靠、连贯知识编辑中的关键作用。代码已公开于 https://github.com/GY-Jeong/STEAM。

原文摘要 · Abstract (English)

Large Language Models store extensive factual knowledge acquired during large-scale pre-training. However, this knowledge is inherently static, reflecting only the state of the world at the time of training. Knowledge editing has emerged as a promising solution for updating outdated or incorrect facts without full retraining. However, most existing locate-and-edit methods primarily focus on token-level likelihood optimization without addressing semantic coherence. Our analysis reveals that such edited knowledge is often encoded as isolated residual streams in the model's latent space, distinct from pre-existing knowledge and bypassing natural reasoning process. To address this, we propose \textsc{Steam}, a semantic-level knowledge editing framework that enhances integration of updated knowledge into the model's knowledge structure. \textsc{Steam} first identifies target representations as semantic anchors for the updated factual association, then guides the internal representation of the edited fact towards these anchors through an alignment loss during optimization. Experimental results demonstrate that \textsc{Steam} improves model's ability to reason with edited knowledge and enhances semantic coherence, underscoring the importance of latent-space alignment for reliable and coherent knowledge editing. The code is available at https://github.com/GY-Jeong/STEAM.

知识编辑语义对齐大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。