让多模态大模型可靠地记住新知识,避免推理时自相矛盾。
CRANE: Knowledge Editing for Reasoning MLLMs

- 用检索增强框架实现无参数修改的知识注入
- 在复杂推理场景中达成96.9%的准确率与97.6%文本定位性
- 适合需要稳定推理链的医疗、金融等高可靠性场景
推理型多模态大模型(MLLMs)在生成答案前会输出显式的思维链(CoT),这带来了新的知识编辑挑战:传统指标下表现良好(教师强制准确率高达100%)的方法,在考察推理过程时可能失效(真实成功度低至0%)。我们识别出三种失败模式:(1)结构坍塌,权重修改破坏了思维链格式;(2)认知冲突,模型基于视觉证据主动排斥新知识;(3)浅层内化,仅对原句查询有效,对改写或多跳问题失败。这些模式相互作用:能泛化的方法(如微调、LoRA)引发格式崩溃,而无深度修改的方法无法泛化。为此,我们提出一种面向思维链的评估协议,并构建ReasonEdit-Bench,包含冲突分层、多级探针和多跳可迁移性测试。提出CRANE框架,无需每次编辑修改参数。它结合模态感知双库检索与两阶段训练策略:先通过监督微调(SFT)初始化结构,再通过GRPO与认知路由奖励训练模型在视觉先验与新知识间权衡。在ReasonEdit-Bench上,CRANE在冲突场景达96.9%真实成功率,多跳链中中间实体使用率达96.9%,文本定位性97.6%,图像定位性68.1%。在分布外的MMEVOKE基准上,黄金检索条件下达到87.0%。
原文摘要 · Abstract (English)
The emergence of reasoning multimodal large language models (MLLMs), which generate explicit chain-of-thought (CoT) reasoning before producing answers, has introduced a new challenge for knowledge editing: methods that appear successful under traditional metrics (teacher-forcing accuracy up to 100%) can fail severely when the model's reasoning process is examined (Grounded Success as low as 0%). We identify three failure modes: (1) Structural Collapse, where weight-modifying methods destroy the CoT format; (2) Cognitive Dissonance, where the model's reasoning chain actively rejects the injected edit fact based on visual evidence; and (3) Shallow Internalization, where methods succeed on exact queries but fail on rephrase or multi-hop variants. On reasoning MLLMs, these modes interact: methods that generalize (FT, LoRA) trigger format collapse, while methods without deep modification cannot generalize. To expose these failures, we propose a CoT-aware evaluation protocol and construct ReasonEdit-Bench, with conflict stratification, multi-level probes, and multi-hop portability tests. We propose CRANE, a retrieval-augmented framework that requires no per-edit parameter modification. CRANE combines a modality-aware dual-library retrieval system with a two-phase training strategy: Supervised Fine-Tuning (SFT) for structural initialization, followed by GRPO with a Cognitive Routing Reward that trains the model to arbitrate between visual priors and injected edit facts. On ReasonEdit-Bench, CRANE achieves 96.9% Grounded Success on conflict scenarios and 96.9% intermediate entity usage in multi-hop chains, with 97.6% text-locality and 68.1% image-locality Edit Independence. On the out-of-distribution MMEVOKE benchmark, CRANE reaches 87.0% under gold retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。