arXiv:2508.01669cs.LGcs.DC2025-08中稿 · ICLR被引 3

用生成模型合成数据,提升异构联邦学习的泛化能力

Bridging Generalization Gap of Heterogeneous Federated Clients Using Generative Models

  • 客户端共享特征分布统计量,不传参数
  • 生成合成数据后微调本地模型,泛化准确率显著提升
  • 适合模型架构不同的分布式场景,通信开销低

联邦学习(FL)是一种保护隐私的分布式机器学习框架,可在多个客户端间协同训练。然而,参与方之间的数据异构性常导致本地模型泛化能力差。传统方法通过正则化或动态加权聚合来缓解此问题,但在客户端模型结构异构时失效。本文提出一种模型异构的联邦学习框架,客户端不传递模型参数,而是共享特征分布的均值与协方差统计量。服务器据此生成高斯潜变量,客户端使用变分逆卷积神经网络生成合成数据,并以此微调本地模型。实验表明,该方法在未见数据上显著提升泛化性能,同时降低通信成本与内存消耗。

原文摘要 · Abstract (English)

Federated Learning (FL) is a privacy-preserving machine learning framework facilitating collaborative training across distributed clients. However, its performance is often compromised by data heterogeneity among participants, which can result in local models with limited generalization capability. Traditional model-homogeneous approaches address this issue primarily by regularizing local training procedures or dynamically adjusting client weights during aggregation. Nevertheless, these methods become unsuitable in scenarios involving clients with heterogeneous model architectures. In this paper, we propose a model-heterogeneous FL framework that enhances clients' generalization performance on unseen data without relying on parameter aggregation. Instead of model parameters, clients share feature distribution statistics (mean and covariance) with the server. Then each client trains a variational transposed convolutional neural network using Gaussian latent variables sampled from these distributions, and use it to generate synthetic data. By fine-tuning local models with the synthetic data, clients achieve significant improvement of generalization ability. Experimental results demonstrate that our approach not only attains higher generalization accuracy compared to existing model-heterogeneous FL frameworks, but also reduces communication costs and memory consumption.

联邦学习生成模型异构

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。