专为罕见病设计的AI模型,提升诊疗信息精准度。
Zebra-Llama: A Context-Aware Large Language Model for Democratizing Rare Disease Knowledge
- 基于医学文献与患者经验微调,增强上下文理解能力
- 在真实病例问答中准确率提升至83.0%,优于基线模型
- 开源框架可推广至其他罕见病,助力医疗公平
罕见病因诊断延迟和信息碎片化面临严峻挑战,其稀缺且可靠的知识数据对大型语言模型(LLMs)构成独特考验。本文提出Zebra-Llama,一种面向埃勒斯-当洛综合征(EDS,发病率1/5000)的上下文感知型大语言模型,具备高精度检索增强生成(RAG)能力。通过结合医学文献、患者经历与临床资源中的问题及专家标注回答,采用创新的上下文感知微调方法进行训练。在由患者与临床医生提供的真实问题测试集上,医疗专家评估显示,相较于基线模型Llama 3.1-8B-Instruct,Zebra-Llama在全面性(77.5% vs. 70.1%)、准确性(83.0% vs. 78.8%)、清晰度(74.7% vs. 72.0%)和引用可靠性(70.6% vs. 52.3%)方面均有显著提升。该模型已开源,不仅为患者提供更可及、可靠的EDS信息,也为开发其他罕见病的专用AI解决方案提供了可复用框架,推动罕见病管理领域专家知识的民主化。
原文摘要 · Abstract (English)
Rare diseases present unique challenges in healthcare, often suffering from delayed diagnosis and fragmented information landscapes. The scarcity of reliable knowledge in these conditions poses a distinct challenge for Large Language Models (LLMs) in supporting clinical management and delivering precise patient information underscoring the need for focused training on these 'zebra' cases. We present Zebra-Llama, a specialized context-aware language model with high precision Retrieval Augmented Generation (RAG) capability, focusing on Ehlers-Danlos Syndrome (EDS) as our case study. EDS, affecting 1 in 5,000 individuals, exemplifies the complexities of rare diseases with its diverse symptoms, multiple subtypes, and evolving diagnostic criteria. By implementing a novel context-aware fine-tuning methodology trained on questions derived from medical literature, patient experiences, and clinical resources, along with expertly curated responses, Zebra-Llama demonstrates unprecedented capabilities in handling EDS-related queries. On a test set of real-world questions collected from EDS patients and clinicians, medical experts evaluated the responses generated by both models, revealing Zebra-Llama's substantial improvements over base model (Llama 3.1-8B-Instruct) in thoroughness (77.5% vs. 70.1%), accuracy (83.0% vs. 78.8%), clarity (74.7% vs. 72.0%) and citation reliability (70.6% vs. 52.3%). Released as an open-source resource, Zebra-Llama not only provides more accessible and reliable EDS information but also establishes a framework for developing specialized AI solutions for other rare conditions. This work represents a crucial step towards democratizing expert-level knowledge in rare disease management, potentially transforming how healthcare providers and patients navigate the complex landscape of rare diseases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。