在消费级显卡上实现医疗病历结构检索,验证本地大模型可行性
GraphRAG on Consumer Hardware: Benchmarking Local LLMs for Healthcare EHR Schema Retrieval

- 用本地部署的开源大模型跑图结构检索,测试四款模型在真实病历数据上的表现
- 70亿参数以下模型难以完成任务,小模型易出错且生成重复内容
- 本地检索比全局摘要更准更快,适合对隐私和合规要求高的医疗场景
基于图的检索增强生成(GraphRAG)拓展了检索增强生成能力,支持在复杂语料中进行结构化推理,但在资源受限、隐私敏感的部署环境下其可靠性仍不明确。在医疗领域,电子健康记录(EHR)数据复杂且受严格监管,依赖云端大语言模型(LLM)带来成本高、延迟大和合规风险。本文系统评估了在本地部署的开源LLM上实现EHR模式检索的GraphRAG性能。我们在真实EHR文档上实现微软GraphRAG流程,使用Llama 3.1(8B)、Mistral(7B)、Qwen 2.5(7B)和Phi-4-mini(3.8B)四款模型,通过Ollama部署于单张8GB VRAM的消费级显卡。评估指标包括索引效率、知识图谱构建、查询延迟、答案质量与幻觉率,涵盖全局与本地检索模式。结果显示:Llama 3.1构建的知识图谱最丰富(1,172个实体),Qwen 2.5答案质量最佳(3.3/5),Phi-4-mini因结构化输出错误无法完成流程,Mistral出现退化重复行为。进一步发现,模型参数量低于约70亿时难以稳定生成有效结构化输出,无法完成全流程;索引与答案质量在不同模型间解耦,且本地检索在延迟和事实准确率上均优于全局摘要,幻觉更少。结果表明,GraphRAG可在消费级硬件上实现,但模型选择与检索设计对合规环境中的可靠部署至关重要。
原文摘要 · Abstract (English)
Graph-based Retrieval Augmented Generation (GraphRAG) extends retrieval-augmented generation to support structured reasoning over complex corpora, but its reliability under resource-constrained, privacy-sensitive deployments remains unclear. In healthcare, where Electronic Health Record (EHR) data is complex and strictly regulated, reliance on cloud-based large language models (LLMs) introduces challenges in cost, latency, and compliance. In this work, we present a systematic evaluation of GraphRAG for EHR schema retrieval using locally deployed open-source LLMs. We implement the Microsoft GraphRAG pipeline on real-world EHR schema documentation and benchmark four models, including Llama 3.1 (8B), Mistral (7B), Qwen 2.5 (7B), and Phi-4-mini (3.8B), each deployed via Ollama on a single consumer GPU (8 GB VRAM). We evaluate indexing efficiency, knowledge graph construction, query latency, answer quality, and hallucination under both global and local retrieval modes. Our results reveal substantial differences: Llama 3.1 produces the richest knowledge graph (1,172 entities), Qwen 2.5 achieves the best answer quality (3.3/5), Phi-4-mini fails to complete the pipeline due to structured-output errors, and Mistral exhibits degenerate repetition behavior. We further show that GraphRAG exhibits a practical capacity threshold, where models below approximately 7B parameters fail to reliably produce valid structured outputs and cannot complete the pipeline. In addition, indexing and answer quality are decoupled across models, and local retrieval consistently outperforms global summarization in both latency and factual grounding, with reduced hallucination. These findings demonstrate that GraphRAG is feasible on consumer hardware while highlighting the importance of model selection and retrieval design for robust deployment in regulated settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。