提出可验证的生物医学问答可靠性框架,提升答案可信度与结构化输出能力。
HypothesisMed: Inference-Time Answer Fusion and Structured Hypothesis-Space Reporting for Biomedical Question Answering

- 推理时融合多种提示策略,生成带置信度和空间标签的答案
- 显著降低错误承诺率,实现100%解析率与空间覆盖
- 适合需要高可靠性输出的医疗AI评估与部署场景
基于大模型的生物医学问答通常仅以答案准确率为评价标准,但该指标无法反映模型是否能生成可解析输出、遵循结构化可靠性指令、识别弱答案空间或避免盲目自信。本文提出HypothesisMed,一种用于生物医学多选题问答的推理时可靠性流水线。该方法结合直接推理、思维链、HypothesisMed-v3提示与答案融合,最终答案通过融合确定,HypothesisMed-v3提供SPACE标签(有效、不完整、矛盾)和置信度信息。在MedQA、MedMCQA、PubMedQA各1,000个样本上测试Qwen2.5-7B、Phi-4-mini、DeepSeek-R1-32B、BioMistral-7B,结果表明该流水线在保持甚至超越最佳基线准确率的同时,显著提升解析率与SPACE覆盖率。进一步对Qwen2.5-7B和Phi-4-mini扩展至10,183样本,融合使Phi-4-mini准确率从0.4296提升至0.5192,而Qwen2.5-7B链式推理仍略优;但其融合版本实现完全解析与空间覆盖,并大幅降低错误承诺。12,000样本的SPACE压力测试显示,答案空间诊断仍具挑战,Qwen2.5-7B与Phi-4-mini的SPACE准确率分别为0.3074与0.4168。结果表明:准确率、可解析性、结构化报告、校准行为与错误承诺是可分离的能力。核心贡献非通用最优,而是一个可复现的推理时框架,使生物医学问答模型在结构化可靠性约束下成为可审计的工作流组件。
原文摘要 · Abstract (English)
Biomedical question answering with large language models is commonly evaluated using answer accuracy, but answer accuracy alone does not indicate whether a model can produce parseable outputs, follow structured reliability instructions, recognize weak answer spaces, or avoid confident incorrect commitments. This paper presents HypothesisMed, an inference-time reliability pipeline for biomedical multiple-choice question answering. It combines direct, chain-of-thought, HypothesisMed-v3 prompting, and answer fusion. The final answer is selected by fusion, while HypothesisMed-v3 supplies SPACE labels and confidence information. SPACE labels mark the answer space as VALID, INCOMPLETE, or CONTRADICTED. We evaluate Qwen2.5-7B, Phi-4-mini, DeepSeek-R1-32B, and BioMistral-7B on MedQA, MedMCQA, and PubMedQA using 1,000 examples per dataset. The pipeline improves weighted accuracy over each model's best direct or chain-of-thought baseline while increasing parse and SPACE coverage. We also scale evaluation to Qwen2.5-7B and Phi-4-mini using 10,183 examples per model. Fusion improves Phi-4-mini accuracy from 0.4296 to 0.5192, while Qwen2.5-7B chain-of-thought remains slightly higher in answer accuracy. However, Qwen2.5-7B fusion achieves complete parse and SPACE coverage with much lower false commitment. A 12,000-example SPACE stress test shows answer-space diagnosis remains difficult, with SPACE accuracy of 0.3074 for Qwen2.5-7B and 0.4168 for Phi-4-mini. These results show that answer accuracy, parseability, structured reliability reporting, calibration behavior, and false-commitment behavior are separable capabilities. The main contribution is not a universal state-of-the-art claim, but a reproducible inference-time framework for evaluating biomedical question answering models as auditable workflow components under structured reliability constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。