用生成数据缓解非独立同分布问题,提升联邦扩散模型性能
Federated Learning for Diffusion Models
- 用客户端扩散模型生成近似全局数据的辅助数据
- 多轮聚合后用辅助数据优化全局模型,提升收敛性
- 适合数据异构场景下的生成模型联邦训练
扩散模型是强大的生成模型,可为各类任务生成高度逼真的样本。通常这些模型依赖集中式、独立同分布(IID)的训练数据构建。然而在实际场景中,数据常分布于多个客户端且呈现非独立同分布(non-IID)特征。联邦学习(FL)可利用分布式数据训练扩散模型,但现有方法在non-IID场景下表现不佳。为此,我们提出FedDDPM——基于去噪扩散概率模型的联邦学习方法,利用扩散模型的数据生成能力促进模型训练。具体地,服务器在联邦训练前使用各客户端上传的本地扩散模型生成近似全局数据分布的辅助数据。每轮模型聚合后,服务器进一步利用该辅助数据集优化全局模型,以缓解异构数据对性能的影响。我们提供了FedDDPM的严格收敛性分析,并提出了改进算法FedDDPM+,通过检测慢收敛实例并采用一次性修正策略降低训练开销。实验结果表明,所提算法在MNIST、CIFAR10和CIFAR100数据集上均优于现有最优联邦学习算法。
原文摘要 · Abstract (English)
Diffusion models are powerful generative models that can produce highly realistic samples for various tasks. Typically, these models are constructed using centralized, independently and identically distributed (IID) training data. However, in practical scenarios, data is often distributed across multiple clients and frequently manifests non-IID characteristics. Federated Learning (FL) can leverage this distributed data to train diffusion models, but the performance of existing FL methods is unsatisfactory in non-IID scenarios. To address this, we propose FedDDPM-Federated Learning with Denoising Diffusion Probabilistic Models, which leverages the data generative capability of diffusion models to facilitate model training. In particular, the server uses well-trained local diffusion models uploaded by each client before FL training to generate auxiliary data that can approximately represent the global data distribution. Following each round of model aggregation, the server further optimizes the global model using the auxiliary dataset to alleviate the impact of heterogeneous data on model performance. We provide a rigorous convergence analysis of FedDDPM and propose an enhanced algorithm, FedDDPM+, to reduce training overheads. FedDDPM+ detects instances of slow model learning and performs a one-shot correction using the auxiliary dataset. Experimental results validate that our proposed algorithms outperform the state-of-the-art FL algorithms on the MNIST, CIFAR10 and CIFAR100 datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。