让小模型先答后思，用幻觉反哺精准推理
Can Hallucinations Be Useful? Solving Multi-Hop Questions With SLMs By Chaining System-I/II Reasoning
- 先快速作答（System-I），再基于假设检索证据深度思考（System-II）
- 在多个多跳问答基准上超越传统先思考后回答的方法
- 适合追求高效推理的小模型应用，尤其擅长处理复杂问题
近期对小型语言模型（SLMs）的兴趣增加，因其速度快、性能好且硬件需求低。然而，与大型语言模型（LLMs）相比，SLMs更易产生幻觉，导致早期错误在多步推理中累积，影响最终答案。现有方法通常采用‘先思考再迭代检索’以减少幻觉。我们提出不同思路：认为‘先思考’并非必要，因我们发现SLMs在初始回答时往往准确且自信，且幻觉本身可助力锁定正确答案。因此，我们提出一种认知启发式框架——‘先回答，后推理’：首先让模型快速作答（System-I，零样本），再根据初始假设从知识源检索证据，进行深度思考（System-II）。结合两种思维模式，该方法在多个多跳问答基准上优于传统‘先思考’策略。
原文摘要 · Abstract (English)
Recently, there has been increased interest in Small Language Models (SLMs), which are fast, show good performance, and have lower hardware demands than large language models (LLMs). However, SLMs hallucinate more frequently than LLMs, impacting their ability to solve complex multi-step reasoning problems as early mistakes cascade to the final response. To address this, existing works think-first followed by iterative retrieval to reduce hallucination. We argue that the think-first strategy is not always necessary as we find that: (i) SLMs are often accurately confident in their initial answer and, (ii) hallucinations can actually be beneficial for honing in on the true answer. As such, we position our work as an inversion of this strategy, i.e., answer first-reason later. We propose a cognitively-inspired framework where the model is first allowed to quickly answer the question (System-I (zero-shot)) and then resorts to deeper thinking (System-II) based on evidence retrieved from a knowledge source using the initial hypothesis. By combining System-I and System-II style thinking, we show that our method can outperform prior work that takes the traditional think-first route on various multi-step question-answering benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。