arXiv:2605.29826cs.CLcs.AI2026-05

解决多模态大模型知识编辑中的错乱问题,让修改更精准且不伤及其他信息。

Towards Localized and Disentangled Knowledge Editing for Multimodal Large Language Models

论文配图:Towards Localized and Disentangled Knowledge Editing for Multimodal Large Language Models
图 1 · 摘自论文原文
  • 定位特定知识对应的模型层,分离相关与无关输入
  • 在多个基准上实现90%以上编辑泛化率,同时保持高局部性
  • 适合需要精准更新知识又怕副作用的研究者

现有多模态知识编辑(MKE)方法虽能修正多模态大语言模型(MLLMs)中的过时或错误知识,但存在关键缺陷:虽能有效修改目标事实对,却难以推广到逻辑相关的查询,并常导致与视觉或语义相关但无关的信息被意外修改。我们识别并形式化了两个根本原因:因果错位(限制编辑仅作用于特定样本)和特征纠缠(引发无关信息的意外改动)。为此,我们提出局部化与解耦知识编辑(LDKE)框架,通过定位事实相关模型层并分离目标相关与无关输入,实现精确且可泛化的编辑。该方法包含快速定位模块,用于高效识别并更新关键层;以及解耦分类器,合理路由输入以保护无关知识。在多个基准和不同MLLM上的实验表明,LDKE在传播编辑至相关上下文方面表现优异,同时保持高局部性。

原文摘要 · Abstract (English)

Existing methods in Multimodal Knowledge Editing (MKE) have advanced the ability to correct outdated or inaccurate knowledge in Multimodal Large Language Models (MLLMs). However, they exhibit a critical limitation: while effectively modifying target factual pairs, they fail to generalize edits to logically related queries and often cause unintended alterations to unrelated but visually or semantically linked information. We identify and formalize two underlying failure modes causing this issue: Causal Misalignment, which confines edits to the specific sample, and Feature Entanglement, which causes unintended alterations to coupled but irrelevant information. To address these issues, we propose Localized and Disentangled Knowledge Editing (LDKE), a new framework that achieves precise and generalized editing by localizing fact-specific model layers and disentangling target-relevant inputs from irrelevant ones. Our approach introduces a Fast Localization module to identify and update critical layers efficiently, along with a Disentanglement Classifier that routes inputs appropriately to preserve unrelated knowledge. Extensive experiments across various benchmarks and MLLMs demonstrate that LDKE achieves superior performance in propagating edits to related contexts while maintaining high locality.

知识编辑多模态大模型解耦

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。