提出去中心化医学联邦学习框架,通过专家混合模型提升隐私保护与系统稳定性。
dFLMoE: Decentralized Federated Learning via Mixture of Experts for Medical Data Analysis
- 客户端直接交换轻量级头模型,用自适应MoE融合本地与接收模型。
- 在多种医疗任务上超越现有方法,异构与同质设置下均表现更优。
- 无需中心服务器,抗故障能力强,适合医疗数据分布式场景。
联邦学习在医疗领域应用广泛,可在保护患者隐私的同时实现多医疗机构间知识共享。然而,现有系统多为集中式,需客户端将特定知识上传至中心服务器进行聚合,此过程会导致知识在集中整合前已受损,且对中心服务器形成依赖,一旦服务器故障或连接不稳定,将影响训练稳定性。为此,本文提出去中心化联邦学习框架dFLMoE:客户端直接交换轻量级头模型,各自将本地及接收的头模型视为独立专家,采用客户端定制的混合专家(MoE)机制进行联合决策。该设计不仅减少因集中聚合造成的信息损失,还消除对中心服务器的依赖,增强系统鲁棒性。我们在多个医疗任务上验证了该框架,结果表明其在模型同质与异质环境下均显著优于现有先进方法。
原文摘要 · Abstract (English)
Federated learning has wide applications in the medical field. It enables knowledge sharing among different healthcare institutes while protecting patients' privacy. However, existing federated learning systems are typically centralized, requiring clients to upload client-specific knowledge to a central server for aggregation. This centralized approach would integrate the knowledge from each client into a centralized server, and the knowledge would be already undermined during the centralized integration before it reaches back to each client. Besides, the centralized approach also creates a dependency on the central server, which may affect training stability if the server malfunctions or connections are unstable. To address these issues, we propose a decentralized federated learning framework named dFLMoE. In our framework, clients directly exchange lightweight head models with each other. After exchanging, each client treats both local and received head models as individual experts, and utilizes a client-specific Mixture of Experts (MoE) approach to make collective decisions. This design not only reduces the knowledge damage with client-specific aggregations but also removes the dependency on the central server to enhance the robustness of the framework. We validate our framework on multiple medical tasks, demonstrating that our method evidently outperforms state-of-the-art approaches under both model homogeneity and heterogeneity settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。