推理能唤醒大模型隐藏的知识,让简单问题答得更准
Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs
- 用推理过程中的生成文本作为计算缓冲,激活潜在知识
- 生成相关事实可作语义桥梁,提升正确答案召回率
- 识别并过滤含幻觉的推理路径,可直接提升准确率
尽管大语言模型在数学、代码生成和多跳事实问答中自然运用推理,但其对单跳事实问题的作用仍不明确。这类问题无需分步逻辑推导,推理的效用显得反直觉。然而我们发现,启用推理显著扩展了模型参数知识的召回边界,解锁了原本无法获取的正确答案。为探究原因,我们设计一系列假设驱动的受控实验,发现两个关键机制:(1) 计算缓冲效应,即模型利用生成的推理标记进行独立于语义内容的隐式计算;(2) 事实提示效应,即生成主题相关事实作为语义桥梁,促进正确答案检索。值得注意的是,该生成自检索机制存在内在风险:推理中幻觉中间事实会增加最终答案的幻觉概率。最后,我们证明可通过优先选择无幻觉事实的推理路径来直接提升模型准确性。
原文摘要 · Abstract (English)
While reasoning in LLMs plays a natural role in math, code generation, and multi-hop factual questions, its effect on simple, single-hop factual questions remains unclear. Such questions do not require step-by-step logical decomposition, making the utility of reasoning highly counterintuitive. Nevertheless, we find that enabling reasoning substantially expands the capability boundary of the model's parametric knowledge recall, unlocking correct answers that are otherwise effectively unreachable. Why does reasoning aid parametric knowledge recall when there are no complex reasoning steps to be done? To answer this, we design a series of hypothesis-driven controlled experiments, and identify two key driving mechanisms: (1) a computational buffer effect, where the model uses the generated reasoning tokens to perform latent computation independent of their semantic content; and (2) factual priming, where generating topically related facts acts as a semantic bridge that facilitates correct answer retrieval. Importantly, this latter generative self-retrieval mechanism carries inherent risks: we demonstrate that hallucinating intermediate facts during reasoning increases the likelihood of hallucinations in the final answer. Finally, we show that our insights can be harnessed to directly improve model accuracy by prioritizing reasoning trajectories that contain hallucination-free factual statements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。