让大模型学会在信息不足时主动暂停,避免胡编乱造。
Pause or Fabricate? Training Language Models for Grounded Reasoning

- 分两阶段:先判断信息是否足够,再进行推理
- 在两个数据集上提升30%任务成功率,减少20%回答长度
- 适合需要可靠推理的AI系统开发者
大型语言模型在复杂推理任务上已取得显著进展,但当输入信息不完整时,常会隐性编造内容,生成看似自信却不可靠的结论——我们称之为非基础推理。我们认为问题根源并非推理能力不足,而是缺乏推断边界意识,即识别必要前提缺失的能力。为此,提出基于交互式强化学习的基底推理框架GRIL,将推理过程分为澄清与暂停、基底推理两个阶段:前者判断信息是否充分,后者在前提完备后执行任务求解。设计阶段特异性奖励,惩罚幻觉,使模型能主动识别知识缺口,暂停并等待澄清后继续。在GSM8K-Insufficient和MetaMATH-Insufficient数据集上的实验表明,GRIL显著提升前提检测能力(最高达45%),任务成功率提高30%,平均响应长度减少超20%。额外分析验证了对噪声用户回复的鲁棒性,并展现出对分布外任务的良好泛化能力。
原文摘要 · Abstract (English)
Large language models have achieved remarkable progress on complex reasoning tasks. However, they often implicitly fabricate information when inputs are incomplete, producing confident but unreliable conclusions -- a failure mode we term ungrounded reasoning. We argue that this issue arises not from insufficient reasoning capability, but from the lack of inferential boundary awareness -- the ability to recognize when the necessary premises for valid inference are missing. To address this issue, we propose Grounded Reasoning via Interactive Reinforcement Learning (GRIL), a multi-turn reinforcement learning framework for grounded reasoning under incomplete information. GRIL decomposes the reasoning process into two stages: clarify and pause, which identifies whether the available information is sufficient, and grounded reasoning, which performs task solving once the necessary premises are established. We design stage-specific rewards to penalize hallucinations, enabling models to detect gaps, stop proactively, and resume reasoning after clarification. Experiments on GSM8K-Insufficient and MetaMATH-Insufficient show that GRIL significantly improves premise detection (up to 45%), leading to a 30% increase in task success while reducing average response length by over 20%. Additional analyses confirm robustness to noisy user responses and generalization to out-of-distribution tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。