arXiv:2412.18460cs.LGcs.AI2024-12被引 3

用生成模型实现异构设备间的联邦学习,保护隐私且无需统一模型结构

GeFL: Model-Agnostic Federated Learning with Generative Models

  • 通过联邦训练生成模型共享全局语义知识
  • 在异构模型下仍保持良好性能,准确率接近同构设置
  • 新方法解决生成样本记忆问题,适合资源受限的边缘设备

联邦学习(FL)是一种分布式训练范式,允许客户端在不共享本地数据的前提下协同学习,从而保护隐私。然而,现代深度模型日益庞大复杂,常超出边缘设备的计算或内存能力。此外,由于硬件差异(如ASIC、FPGA)或专有要求,客户端可能被迫使用异构模型架构,无法修改或披露本地模型结构。这推动了对异构模型联邦学习的需求。本文提出生成式模型辅助联邦学习(GeFL),通过联邦训练的生成模型实现跨客户端的知识共享。该生成模型捕捉全局数据语义,使本地训练无需客户端模型同质化。尽管GeFL表现优异,但实验显示其在可扩展性及隐私泄露方面存在局限,源于生成样本的记忆现象。为此,我们提出GeFL-F,采用特征级生成建模,提升大规模客户端的可扩展性并缓解隐私风险。在图像分类任务上的大量实验表明,两者在异构场景下均具有竞争力。代码已公开。

原文摘要 · Abstract (English)

Federated learning (FL) is a distributed training paradigm that enables collaborative learning across clients without sharing local data, thereby preserving privacy. However, the increasing scale and complexity of modern deep models often exceed the computational or memory capabilities of edge devices. Furthermore, clients may be constrained to use heterogeneous model architectures due to hardware variability (e.g., ASICs, FPGAs) or proprietary requirements that prevent the disclosure or modification of local model structures. These practical considerations motivate the need for model-heterogeneous FL, where clients participate using distinct model architectures. In this work, we propose Generative Model-Aided Federated Learning (GeFL), a framework that enables cross-client knowledge sharing via a generative model trained in a federated manner. This generative model captures global data semantics and facilitates local training without requiring model homogeneity across clients. While GeFL achieves strong performance, empirical analysis reveals limitations in scalability and potential privacy leakage due to generative sample memorization. To address these concerns, we propose GeFL-F, which utilizes feature-level generative modeling. This approach enhances scalability to large client populations and mitigates privacy risks. Extensive experiments across image classification tasks demonstrate that both GeFL and GeFL-F offer competitive performance in heterogeneous settings. Code is available at [1].

联邦学习生成模型异构模型隐私保护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。