用AI在真实医院里自动查病历，准确率近90%以上
EHR-MCP: Real-world Evaluation of Clinical Information Retrieval by Large Language Models via Model Context Protocol
- 通过MCP协议连接大模型与医院病历系统，实现自主检索
- 6项任务中5项准确率接近完美，复杂任务因时间计算出错
- 适合医疗AI研究者和医院信息化团队参考
大型语言模型（LLMs）在医学领域潜力巨大，但因难以接入电子健康记录（EHR）系统而受限。本文提出EHR-MCP框架，通过模型上下文协议（MCP）实现大模型与医院EHR数据库的集成。采用GPT-4.1配合LangGraph ReAct代理，在真实医院环境中执行六项由感染控制团队提出的任务，对八名患者病例进行回溯分析。结果表明，模型能稳定选择并执行正确的MCP工具，除两项任务外其余均达近完美准确率。性能下降主要源于时间依赖性计算的复杂任务，错误多由参数错误或结果误读导致。虽然响应存在冗长重复风险，可能超出上下文窗口，但整体数据输出可靠。研究证明，通过MCP可实现大模型在真实场景下高效获取临床信息，为医院级AI代理提供安全一致的数据访问基础。未来需拓展至推理、生成及临床影响评估，推动生成式AI在医疗实践中的有效落地。
原文摘要 · Abstract (English)
Background: Large language models (LLMs) show promise in medicine, but their deployment in hospitals is limited by restricted access to electronic health record (EHR) systems. The Model Context Protocol (MCP) enables integration between LLMs and external tools. Objective: To evaluate whether an LLM connected to an EHR database via MCP can autonomously retrieve clinically relevant information in a real hospital setting. Methods: We developed EHR-MCP, a framework of custom MCP tools integrated with the hospital EHR database, and used GPT-4.1 through a LangGraph ReAct agent to interact with it. Six tasks were tested, derived from use cases of the infection control team (ICT). Eight patients discussed at ICT conferences were retrospectively analyzed. Agreement with physician-generated gold standards was measured. Results: The LLM consistently selected and executed the correct MCP tools. Except for two tasks, all tasks achieved near-perfect accuracy. Performance was lower in the complex task requiring time-dependent calculations. Most errors arose from incorrect arguments or misinterpretation of tool results. Responses from EHR-MCP were reliable, though long and repetitive data risked exceeding the context window. Conclusions: LLMs can retrieve clinical data from an EHR via MCP tools in a real hospital setting, achieving near-perfect performance in simple tasks while highlighting challenges in complex ones. EHR-MCP provides an infrastructure for secure, consistent data access and may serve as a foundation for hospital AI agents. Future work should extend beyond retrieval to reasoning, generation, and clinical impact assessment, paving the way for effective integration of generative AI into clinical practice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。