arXiv:2602.14189cs.CLcs.AI2026-02被引 3

让大模型学会在不确定时选择不回答,提升科学推理可靠性。

Knowing When Not to Answer: Abstention-Aware Scientific Reasoning

  • 将科学命题拆解为最小条件,用自然语言推理判断每条证据是否支持
  • 信心阈值引导的弃答机制在中等覆盖下显著降低错误率
  • 适合需要高可信度的科研验证场景,尤其对开源与闭源模型都适用

大型语言模型被广泛用于回答和验证科学命题,但现有评估通常假设模型必须始终给出确定答案。在科学领域,未经支持或不确定的结论可能比不回答更具危害性。本文提出一种弃答感知的验证框架:将科学命题分解为最小条件,通过自然语言推理(NLI)逐项审计每项条件的证据支持度,并决定是否支持、反驳或弃答。我们在两个互补的科学基准上评估该框架:SciFact 和 PubMedQA,涵盖闭卷与开域证据设置。实验使用六种不同架构的模型,包括编码器-解码器模型、开源对话模型及专有API。所有基准与模型中,原始准确率在不同架构间差异不大,而弃答策略对控制错误至关重要。特别是基于置信度的弃答机制在中等覆盖水平下显著降低风险,即使绝对准确率提升有限。结果表明,在科学推理任务中,核心挑战并非选择最优模型,而是判断现有证据是否足以支撑答案。本工作强调弃答感知评估作为衡量科学可靠性的实用且模型无关视角,并为未来科学领域选择性推理研究提供统一实验基础。代码已公开于 https://github.com/sabdaljalil2000/ai4science。

原文摘要 · Abstract (English)

Large language models are increasingly used to answer and verify scientific claims, yet existing evaluations typically assume that a model must always produce a definitive answer. In scientific settings, however, unsupported or uncertain conclusions can be more harmful than abstaining. We study this problem through an abstention-aware verification framework that decomposes scientific claims into minimal conditions, audits each condition against available evidence using natural language inference (NLI), and selectively decides whether to support, refute, or abstain. We evaluate this framework across two complementary scientific benchmarks: SciFact and PubMedQA, covering both closed-book and open-domain evidence settings. Experiments are conducted with six diverse language models, including encoder-decoder, open-weight chat models, and proprietary APIs. Across all benchmarks and models, we observe that raw accuracy varies only modestly across architectures, while abstention plays a critical role in controlling error. In particular, confidence-based abstention substantially reduces risk at moderate coverage levels, even when absolute accuracy improvements are limited. Our results suggest that in scientific reasoning tasks, the primary challenge is not selecting a single best model, but rather determining when available evidence is sufficient to justify an answer. This work highlights abstention-aware evaluation as a practical and model-agnostic lens for assessing scientific reliability, and provides a unified experimental basis for future work on selective reasoning in scientific domains. Code is available at https://github.com/sabdaljalil2000/ai4science .

科学推理弃答机制大模型评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。