让大模型像人一样逐步更新知识，避免自相矛盾。
From Fact Overwriting to Knowledge Evolution: Causal Editing via On-Policy Self-Distillation
- 用因果逻辑替代强行替换，让知识更新更自然。
- 自相矛盾率从95.6%降到1.8%，多跳推理准确率达83.5%。
- 适合需要持续学习、避免知识冲突的研究者。
知识编辑虽能高效更新，但主流的静态事实覆盖范式将大模型视为离散数据库，强制注入孤立事实，破坏预训练逻辑结构，引发认知失调——旧知识优先导致模型需显式否定新信息。理想干预显示此为结构性缺陷，零扰动代理下自相矛盾率高达95.6%。基于真实世界知识的因果性，以显式因果叙事为基础可将冲突率降至6.6%。为此提出CODE（因果在线策略蒸馏编辑），结合因果自举与非对称在线策略蒸馏，将因果演化逻辑直接嵌入参数记忆。在LLaMA-3.1和Qwen-2.5上的实验表明，CODE将自相矛盾率压至1.8%，同时保持强多跳推理能力（最高达83.5%），实现从离散事实注入到连贯知识演进的转变。代码已开源。
原文摘要 · Abstract (English)
While Knowledge Editing (KE) enables efficient updates, its dominant Static Fact Overwriting paradigm treats LLMs as discrete databases, forcibly injecting isolated facts. Fracturing pre-trained logical topologies, this triggers Epistemic Dissonance -- a pathology where un-evolved legacy priors force the model to explicitly negate the injected update. Idealized interventions reveal that this is an inherent structural flaw rather than mere algorithmic noise, with a zero-distortion proxy yielding a catastrophic 95.6% self-refutation rate. Given the causally driven nature of real-world knowledge, grounding updates in explicit causal narratives effectively collapses this conflict rate to just 6.6%, underscoring the imperative for a paradigm shift toward Causal Editing. To internalize this evolution, we propose CODE (Causal On-policy Distillation for Editing). By coupling causal bootstrapping with asymmetric on-policy distillation, CODE engraves causal transition logic directly into parametric memory. Experiments on LLaMA-3.1 and Qwen-2.5 show CODE drastically suppresses self-refutation to 1.8% while securing robust multi-hop accuracy (up to 83.5%), seamlessly transforming discrete fact injection into coherent knowledge evolution. Code is available at https://github.com/CrashBugger/CODE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。