用轻量适配器实现多机构医疗大模型协作,保护隐私且效果更好
Toward Federated Large Language Models in Medicine: A Parameter-Efficient Framework for Privacy-Preserving, Multi-Institutional Adaptation
- 只传低秩适配器,大幅降低通信开销
- 在5个医院数据集上实体与关系提取准确率显著提升
- 适合资源少的新机构快速部署,特别适合跨院数据协作
大型语言模型(LLM)在医疗领域应用日益广泛,但因隐私和治理限制,多数模型仅基于单机构数据训练,导致在异构医疗系统间泛化能力差。为此,我们提出 Fed-MedLoRA 与 Fed-MedLoRA+,一种参数高效的联邦学习框架,支持多机构协作适应医疗LLM。Fed-MedLoRA 仅传输低秩适配器而非完整模型权重,降低通信开销;还评估了对适配器更新添加高斯扰动的隐私保护变体。Fed-MedLoRA+ 进一步引入自适应聚合机制,缓解患者群体、标注习惯与疾病分布的跨站点差异。我们在包含42,198个实体和41,570个关系的五个独立患者队列上进行评估,对比零样本、微调的LLM、领域专用BERT模型及联邦基线。所有设置下,新方法均持续提升抽取性能并更好泛化至异构队列。在耶鲁纽黑文医疗系统的真实案例研究中,该框架在低资源新机构部署时表现优异。结果表明,联邦化、参数高效的医疗大模型适配在多机构临床部署中可行、可扩展且有效。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly adapted for medical applications, but most are trained using data from a single institution because privacy and governance constraints prevent multi-institutional data sharing. As a result, these models often generalize poorly across heterogeneous healthcare systems. We address this gap by introducing Fed-MedLoRA and Fed-MedLoRA+, a parameter-efficient federated framework for collaborative LLM adaptation across healthcare institutions. Fed-MedLoRA transmits only low-rank adapters rather than full model weights, reducing communication overhead. We also evaluate a privacy-preserving variant that applies Gaussian perturbation to transmitted adapter updates. Fed-MedLoRA+ further incorporates adaptive aggregation to better address cross-site heterogeneity in patient populations, annotation practices, and disease distributions. We evaluate the framework on clinical information extraction across five independent patient cohorts totaling 42,198 entities and 41,570 relations, and compare it with zero-shot and fine-tuned LLMs, domain-specific BERT models, and federated baselines. Across all settings, the proposed methods consistently improve extraction performance and generalize better to heterogeneous cohorts. In a real-world case study using clinical notes from the Yale New Haven Health System, the framework demonstrates strong performance under low-resource new-site deployment. These results suggest that federated, parameter-efficient LLM adaptation is feasible, scalable, and effective for multi-institutional clinical deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。