arXiv:2604.00261cs.CL2026-04被引 1

测试大模型自我纠错在医学问答中的效果,发现不总是有效。

Can Large Language Models Self-Correct in Medical Question Answering? An Exploratory Study

  • 用迭代反思机制让模型自己检查并修改推理过程。
  • 在MedQA上略有提升,但在其他数据集上效果有限甚至变差。
  • 适合研究模型决策过程,不适合直接提升医疗问答可靠性。

大型语言模型(LLMs)在医学问答(Medical QA)任务中表现强劲,链式思维(CoT)提示进一步通过激发显式中间推理提升了效果;与此同时,自我反思(self-corrective)提示被广泛认为可通过引导模型自我批评和修正推理来增强可靠性,但其在高安全要求的医疗场景中的有效性尚不明确。本文对医学多选题问答中的自我反思推理进行探索性分析:使用GPT-4o和GPT-4o-mini,在三个主流医学QA基准(MedQA、HeadQA、PubMedQA)上比较标准CoT提示与迭代反思循环,追踪预测在反思步骤间的演化情况。分析自我反思是否带来错误修正、错误持续或引入新错误。结果表明,自我反思提示并未一致提升准确率,其效果高度依赖数据集与模型:在MedQA上取得微弱增益,但在HeadQA和PubMedQA上收益有限或为负,增加反思步数也不保证性能提升。该发现揭示了推理透明性与推理正确性之间的差距,提示自我反思更适合作为理解模型行为的分析工具,而非提升医学QA可靠性的独立解决方案。

原文摘要 · Abstract (English)

Large language models (LLMs) have achieved strong performance on medical question answering (medical QA), and chain-of-thought (CoT) prompting has further improved results by eliciting explicit intermediate reasoning; meanwhile, self-reflective (self-corrective) prompting has been widely claimed to enhance model reliability by prompting LLMs to critique and revise their own reasoning, yet its effectiveness in safety-critical medical settings remains unclear. In this work, we conduct an exploratory analysis of self-reflective reasoning for medical multiple-choice question answering: using GPT-4o and GPT-4o-mini, we compare standard CoT prompting with an iterative self-reflection loop and track how predictions evolve across reflection steps on three widely used medical QA benchmarks (MedQA, HeadQA, and PubMedQA). We analyze whether self-reflection leads to error correction, error persistence, or the introduction of new errors. Our results show that self-reflective prompting does not consistently improve accuracy and its impact is highly dataset- and model-dependent: it yields modest gains on MedQA but provides limited or negative benefits on HeadQA and PubMedQA, and increasing the number of reflection steps does not guarantee better performance. These findings highlight a gap between reasoning transparency and reasoning correctness, suggesting that self-reflective reasoning is better viewed as an analytical tool for understanding model behavior rather than a standalone solution for improving medical QA reliability.

医学问答自我纠错大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。