发现大模型记住了知识却不会用，提出诊断与修复方法
Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning
- 用自修补技术定位知识在模型中的激活位置
- 发现记忆与推理间存在准确率差距和延迟，修复后提升58%~75%
- 适合研究模型泛化能力或微调优化的研究者
将大语言模型微调以注入新知识时面临核心挑战：模型能快速记忆新事实，却难以用于下游推理任务。我们将其失败定义为‘知-用差距’，表现为记忆与泛化间的准确率差距及时间滞后。通过引入新型干预技术‘自修补’，在微调过程中监控知识在模型内部的传播动态。该方法识别出对推理失败案例有显著改善作用的激活位置，结果支持‘知识回路错位’假说——记忆表征虽存在于模型内部，但未被路由至有效计算层。为验证诊断有效性，设计简单启发式策略，成功恢复了58%至75%的基准性能余量。跨领域实验验证了该现象的稳健性。
原文摘要 · Abstract (English)
Fine-tuning LLMs to inject new knowledge faces a critical challenge: LLMs can quickly memorize new facts, yet fail to use them for downstream reasoning tasks. We formalize this failure as the \textit{\textbf{Knowing--Using Gap}}, characterized by an accuracy gap and a temporal lag between memorization and generalization. To understand this phenomenon, we fine-tune LLMs with unseen knowledge and monitor the spatial permeation dynamics of the knowledge internally using a novel intervention technique called self-patching. Self-patching identifies activation locations where relocating representations substantially improves failed generalization cases. These results are consistent with a knowledge-circuit misalignment hypothesis: memorized representations can exist internally but may not be routed to computation-effective layers. To demonstrate the practicality of this diagnostic finding, we design a simple heuristic strategy which recovers 58--75\% of the oracle headroom in generalization failure. Experiments are done cross-domain for the robustness of this finding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。