arXiv:2602.04755cs.CLcs.AI2026-02中稿 · ICLR被引 5

让大模型学会在不确定时沉默,提升时间问答可靠性。

When Silence Is Golden: Can LLMs Learn to Abstain in Temporal QA and Beyond?

  • 用思维链监督+强化学习,教模型在不确定时选择不回答。
  • 在TimeQA数据集上,模型超越GPT-4o 3.46%~5.80%准确率,未答对题识别率提升20%。
  • 发现显式思维链对沉默训练帮助有限,强化学习更有效,适合追求可靠性的场景。

大语言模型常在不确定时强行生成答案,而非选择不回答,这一缺陷在时间问答中尤为明显,模型常忽略时间敏感信息并混淆不同时期事实。本文首次实证研究在时间问答中训练模型具备拒绝回答能力的方法。不同于依赖校准的旧方法,我们将其视为可教学技能,提出结合思维链(CoT)监督与基于拒答感知奖励的强化学习(RL)框架。通过大量实验分析不同信息类型和训练策略的影响,发现使用初始为Qwen2.5-1.5B-Instruct的模型,在TimeQA-Easy和Hard上分别超越GPT-4o 3.46%和5.80%的精确匹配率;在无法回答的问题上,真阳性率比纯监督微调(SFT)版本提升20%。分析显示,SFT会引发过度自信,损害可靠性;而RL虽提高准确率,仍存在类似风险。对比隐含线索(如上下文、时间子上下文、知识图谱)与显式思维链发现,前者对拒答推理帮助甚微。本研究为联合优化拒答与推理提供了新思路,奠定构建更可靠大模型的基础。

原文摘要 · Abstract (English)

Large language models (LLMs) rarely admit uncertainty, often producing fluent but misleading answers, rather than abstaining (i.e., refusing to answer). This weakness is even evident in temporal question answering, where models frequently ignore time-sensitive evidence and conflate facts across different time-periods. In this paper, we present the first empirical study of training LLMs with an abstention ability while reasoning about temporal QA. Existing approaches such as calibration might be unreliable in capturing uncertainty in complex reasoning. We instead frame abstention as a teachable skill and introduce a pipeline that couples Chain-of-Thought (CoT) supervision with Reinforcement Learning (RL) guided by abstention-aware rewards. Our goal is to systematically analyze how different information types and training techniques affect temporal reasoning with abstention behavior in LLMs. Through extensive experiments studying various methods, we find that RL yields strong empirical gains on reasoning: a model initialized by Qwen2.5-1.5B-Instruct surpasses GPT-4o by $3.46\%$ and $5.80\%$ in Exact Match on TimeQA-Easy and Hard, respectively. Moreover, it improves the True Positive rate on unanswerable questions by $20\%$ over a pure supervised fine-tuned (SFT) variant. Beyond performance, our analysis shows that SFT induces overconfidence and harms reliability, while RL improves prediction accuracy but exhibits similar risks. Finally, by comparing implicit reasoning cues (e.g., original context, temporal sub-context, knowledge graphs) with explicit CoT supervision, we find that implicit information provides limited benefit for reasoning with abstention. Our study provides new insights into how abstention and reasoning can be jointly optimized, providing a foundation for building more reliable LLMs.

大模型拒绝回答时间问答强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。