优化提示词可显著提升大模型在病历错误检测中的表现
Importance of Prompt Optimisation for Error Detection in Medical Notes Using Language Models
- 用遗传-帕累托算法自动优化提示词，提升模型判断力
- 在MEDEC数据集上准确率从0.669提升至0.785（GPT-5）
- 适合医疗AI研发者、临床信息处理研究人员参考
医疗文本中的错误可能导致患者诊疗延误甚至误治。近年来，语言模型在自动检测医疗文本错误方面展现出潜力，有望显著改善医疗系统。本文研究了提示词优化对小规模与大规模语言模型在错误检测任务中的影响。我们在前沿语言模型与开源模型上进行了严谨实验与分析。结果表明，采用遗传-帕累托（GEPA）算法进行自动提示词优化，使GPT-5的错误检测准确率从0.669提升至0.785，Qwen3-32B从0.578提升至0.690，接近医生水平，并在MEDEC基准数据集上达到当前最佳性能。代码已公开于GitHub：https://github.com/CraigMyles/clinical-note-error-detection。
原文摘要 · Abstract (English)
Errors in medical text can cause delays or even result in incorrect treatment for patients. Recently, language models have shown promise in their ability to automatically detect errors in medical text, an ability that has the opportunity to significantly benefit healthcare systems. In this paper, we explore the importance of prompt optimisation for small and large language models when applied to the task of error detection. We perform rigorous experiments and analysis across frontier language models and open-source language models. We show that automatic prompt optimisation with Genetic-Pareto (GEPA) improves error detection over the baseline accuracy performance from 0.669 to 0.785 with GPT-5 and 0.578 to 0.690 with Qwen3-32B, approaching the performance of medical doctors and achieving state-of-the-art performance on the MEDEC benchmark dataset. Code available on GitHub: https://github.com/CraigMyles/clinical-note-error-detection
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。