通过模块化融合实现多机构协作训练医疗大模型,保护患者隐私。
MedForge: Building Medical Foundation Models Like Open Source Software Development
- 采用低秩适配模块异步融合,支持机构间安全协作建模。
- 在乳腺癌、肺癌等多中心数据集上表现优异,性能持续提升。
- 适合医疗数据孤岛严重场景下的跨机构模型共建团队。
基础模型(FMs)已在医疗领域取得显著进展,但医疗系统中的数据孤岛和隐私问题仍阻碍安全的数据共享与协同模型开发。构建可扩展的临床数据集已成为训练强健基础模型的主要瓶颈。本文提出医疗基础模型融合框架 MedForge,一种社区驱动的协作机制,可在防止原始患者数据泄露的同时,缓解不同医疗机构间模型同步开发的问题。MedForge 采用自下而上的模型构建方式,灵活合并任务特定的低秩适配(LoRA)模块,在保留原始模型参数的前提下适应下游任务。通过异步 LoRA 模块集成方案,复合模型可逐步提升在多种临床任务上的综合性能。MedForge 在多个来自不同机构的临床数据集(如乳腺癌、肺癌、结肠癌)上表现出色。主要发现表明,协作式基础模型能有效促进多中心临床合作的深入与协调。代码已公开于 https://github.com/TanZheling/MedForge。
原文摘要 · Abstract (English)
Foundational models (FMs) have made significant strides in the healthcare domain. Yet the data silo challenge and privacy concern remain in healthcare systems, hindering safe medical data sharing and collaborative model development among institutions. The collection and curation of scalable clinical datasets increasingly become the bottleneck for training strong FMs. In this study, we propose Medical Foundation Models Merging (MedForge), a cooperative framework enabling a community-driven medical foundation model development, meanwhile preventing the information leakage of raw patient data and mitigating synchronization model development issues across clinical institutions. MedForge offers a bottom-up model construction mechanism by flexibly merging task-specific Low-Rank Adaptation (LoRA) modules, which can adapt to downstream tasks while retaining original model parameters. Through an asynchronous LoRA module integration scheme, the resulting composite model can progressively enhance its comprehensive performance on various clinical tasks. MedForge shows strong performance on multiple clinical datasets (e.g., breast cancer, lung cancer, and colon cancer) collected from different institutions. Our major findings highlight the value of collaborative foundation models in advancing multi-center clinical collaboration effectively and cohesively. Our code is publicly available at https://github.com/TanZheling/MedForge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。