让大模型在不确定时主动放弃，通过精准补救提升医疗判断准确率。
Rethinking LLM Verification: Evidence Structure, Uncertainty, and Selective Refinement
- 模型不确定时放弃判断，仅对放弃项进行针对性知识补全。
- 医疗题库测试中，问题准确率提升9.6个百分点至92.5%。
- 无需构建知识图谱即可达到类似效果，适合高风险场景应用。
大型语言模型常依赖捷径而非系统推理，医疗应用中存在安全风险。允许模型在不确定时放弃判断可提升可靠性，但会带来覆盖率与准确率的权衡。本文提出一种两阶段医学假说验证框架，在多选题设置下通过仅在模型放弃时实施针对性本体锚定来管理该权衡。实验表明，放弃行为并非随机，而是反映真实不确定性，被放弃的预测关联更低置信度。在GPT-5.5（通过Azure OpenAI API访问）和DeepSeek-R1两个前沿模型上，该框架使问题级准确率提升9.6个百分点（82.9%→92.5%），假说级准确率提升4.2个百分点（92.0%→96.2%）。基于MedReason和MedQA的数据实验证明，放弃可作为选择性推理精炼的控制信号，实现接近知识图谱级别的性能，而无需显式构建知识图谱。
原文摘要 · Abstract (English)
Large language models (LLMs) often rely on shortcuts rather than systematic reasoning, raising safety concerns in medical applications. Allowing models to abstain when uncertain improves reliability but introduces a coverage accuracy tradeoff. We propose a two-stage framework for medical hypothesis verification in multiple-choice settings that manages this tradeoff through targeted ontology grounding, applied only when the model abstains. We show that abstention is not random but reflects genuine uncertainty, with abstained predictions associated with lower confidence. Across two frontier models (GPT-5.5, accessed via the Azure OpenAI API, and DeepSeek-R1), the proposed framework improves question-level accuracy by 9.6 percentage points (82.9% to 92.5%) and hypothesis-level accuracy by 4.2 percentage points (92.0% to 96.2%). Our experiments conducted on MedReason and MedQA show that abstention can be repurposed as a control signal for selective reasoning refinement, achieving knowledge-graph-level performance without explicit knowledge graph construction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。