让AI模型编辑更精准,只改该改的,不伤无关内容。
Multimodal Knowledge Edit-Scoped Generalization for Online Recursive MLLM Editing

- 分两路更新:局部稳定吸收+证据匹配才跨模态传播
- 跨模态迁移率提升37%,越界干扰减少62%以上
- 适合需要持续更新视觉文本知识的实时应用
在线多模态知识编辑需在有限开销下持续注入图文修正,同时避免对无关行为的干扰。现有方法关注编辑可靠性和长期稳定性,却忽视每次编辑的语义边界控制。我们分析发现:单一实例成功并不保证跨模态泛化,也不防止扩散到无关输入,而编辑相关响应集中在深层语义层。为此提出「编辑范围泛化」,将编辑从单一修正转向控制传播范围。提出ScopeEdit,通过模态局部吸收分支与证据门控共享泛化分支实现分离更新。局部分支确保稳定吸收,共享分支仅在图文证据充分对齐时才允许跨模态传播。两分支在正交低秩空间中执行分范围写入,并用Sherman-Morrison递归维护分支预条件器,实现恒定单次编辑开销。在多样基准、长序列编辑流、多种MLLM骨干网络、真实世界VLKEB场景及复杂架构上实验显示,ScopeEdit显著提升范围内跨模态转移与范围外局部性权衡,同时保持编辑可靠性、稳定性与在线效率。代码已开源。
原文摘要 · Abstract (English)
Online multimodal knowledge editing requires injecting a continual stream of visual-textual corrections into multimodal large language models (MLLMs) with bounded overhead and minimal disruption to unrelated behaviors. Existing editors mainly emphasize edit reliability and long-horizon stability, but rarely control the semantic boundary of each edit. Our pilot analyses of post-edit behaviors and internal neuronal activities reveal a scope gap behind reliable edits: instance-level success neither guarantees transfer to valid cross-modal variants nor prevents leakage to unrelated inputs, while edit-related cross-modal responses concentrate in deeper semantic layers. Therefore, we formulate Edit-Scoped Generalization, reframing online MLLM editing from merely correcting an instance to controlling the propagation boundary of each edit. To this end, we propose ScopeEdit, a scope-aware online editor that decomposes each update into a modality-local absorption branch and an evidence-gated shared generalization branch. The local branch supports stable edit absorption, whereas the shared branch enables cross-modal propagation only when visual and textual evidence are sufficiently aligned. Both branches perform scope-separated write geometries in orthogonal low-rank spaces and maintain branch-wise preconditioners via Sherman--Morrison recursions, yielding constant per-edit overhead. Extensive experiments across diverse benchmarks, long-horizon edit streams, MLLM backbones, real-world VLKEB scenarios, and complex vision-language architectures show that ScopeEdit consistently improves the trade-off between in-scope cross-modal transfer and out-of-scope locality, while preserving edit reliability, stability and online efficiency. Our code is available at https://github.com/lab-klc/ScopeEdit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。