大模型可自动生成符合临床标准的医学考试题,辅助医学生学习。
The Potential of LLMs in Medical Education: Generating Questions and Answers for Qualification Exams
- 用少量示例提示词让大模型基于真实病历生成考题与答案。
- 8个主流大模型生成内容在逻辑性和专业性上接近临床医生水平。
- 适合医学生、规培生快速理解复杂病例,提升备考效率。
本研究利用大语言模型(LLMs)通过少量示例提示,生成医学资格考试题目及对应答案,深入评估其在连贯性、陈述证据、事实一致性与专业性等方面的表现。基于名为老年共病医学数据库(CECMed)的多中心双向匿名数据库,该数据集包含2010年1月至2022年1月的回顾性队列和2023年1月至11月的前瞻性队列,涵盖中国南、北、中部地区多家三甲及社区医院的患者信息。研究选取了8个广泛使用的大型语言模型(包括ERNIE 4、ChatGLM 4、Doubao、Hunyuan、Spark 4、Qwen等),基于部分抽取的入院记录生成开放式问题与答案。传统医学教育依赖资深医师从电子病历中提炼考题,过程耗时且主观性强。结果显示,主流大模型生成的内容在真实病历基础上达到接近临床医生的水平。尽管在某些方面表现仍不理想,但医学生成、实习医生和住院医师已可合理利用大模型辅助理解与学习。
原文摘要 · Abstract (English)
In this work, we leverage LLMs to produce medical qualification exam questions and the corresponding answers through few-shot prompts, investigating in-depth how LLMs meet the requirements in terms of coherence, evidence of statement, factual consistency, and professionalism etc. Utilizing a multicenter bidirectional anonymized database with respect to comorbid chronic diseases, named Elderly Comorbidity Medical Database (CECMed), we tasked LLMs with generating open-ended questions and answers based on a subset of sampled admission reports. For CECMed, the retrospective cohort includes patients enrolled from January 2010 to January 2022 while the prospective cohort from January 2023 to November 2023, with participants sourced from selected tertiary and community hospitals across the southern, northern, and central regions of China. A total of 8 widely used LLMs were used, including ERNIE 4, ChatGLM 4, Doubao, Hunyuan, Spark 4, Qwen, Conventional medical education requires sophisticated clinicians to formulate questions and answers based on prototypes from EHRs, which is heuristic and time-consuming. We found that mainstream LLMs could generate questions and answers with real-world EHRs at levels close to clinicians. Although current LLMs performed dissatisfactory in some aspects, medical students, interns and residents could reasonably make use of LLMs to facilitate understanding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。