通过正交LoRA适配器实现医疗视觉模型个性化联邦微调
Personalized Federated Fine-Tuning of Vision Foundation Models for Healthcare
- 用正交LoRA适配器分离通用与客户专属知识
- 在真实医疗影像任务中表现优于现有联邦微调方法
- 适合需保护隐私的医院间协作场景
基础模型为医疗AI应用带来新可能，但即便在医疗数据上预训练，仍需针对具体下游任务微调。尽管基础模型降低了对训练数据量的需求，但获取足够数据仍是挑战，部分原因在于不同来源数据共享与聚合受限，以保护患者隐私。一种可行方案是通过跨多个参与方（如医院、诊所）的联邦学习微调基础模型。本文提出一种新的个性化联邦微调方法，通过学习正交的LoRA适配器，解耦通用知识与客户端特定知识，使每个客户端能充分使用自身数据及他人数据。在真实世界联邦医疗影像任务中的初步结果表明，该方法在性能上可与当前主流联邦微调方法竞争。
原文摘要 · Abstract (English)
Foundation models open up new possibilities for the use of AI in healthcare. However, even when pre-trained on health data, they still need to be fine-tuned for specific downstream tasks. Furthermore, although foundation models reduce the amount of training data required to achieve good performance, obtaining sufficient data is still a challenge. This is due, in part, to restrictions on sharing and aggregating data from different sources to protect patients' privacy. One possible solution to this is to fine-tune foundation models via federated learning across multiple participating clients (i.e., hospitals, clinics, etc.). In this work, we propose a new personalized federated fine-tuning method that learns orthogonal LoRA adapters to disentangle general and client-specific knowledge, enabling each client to fully exploit both their own data and the data of others. Our preliminary results on real-world federated medical imaging tasks demonstrate that our approach is competitive against current federated fine-tuning methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。