用扩散模型生成历史数据，缓解联邦学习中的遗忘问题。
Using Diffusion Models as Generative Replay in Continual Federated Learning -- What will Happen?
- 在本地设备上用条件扩散模型生成合成历史数据。
- 在多个数据集上验证，显著减少灾难性遗忘。
- 适合动态数据分布的持续联邦学习场景。
联邦学习（FL）已成为去中心化学习的核心，但在许多场景中，数据分布会随时间动态变化，引发持续学习（CL）问题。这一持续联邦学习（CFL）任务面临独特挑战，尤其是灾难性遗忘和非独立同分布（non-IID）输入数据。现有解决方案包括使用重放缓冲区存储历史数据或利用生成对抗网络。然而，受扩散模型在生成任务中最新进展的启发，本文提出DCFL框架，专为应对动态分布式学习环境中的CFL挑战而设计。该方法在每次通信时，利用条件扩散模型在本地设备生成合成历史数据，有效缓解动态数据分布中的潜在偏移。我们给出了所提CFL框架的收敛界，并在多个数据集上展示了其出色性能，证明了其在应对CFL复杂性方面的有效性。
原文摘要 · Abstract (English)
Federated learning (FL) has become a cornerstone in decentralized learning, where, in many scenarios, the incoming data distribution will change dynamically over time, introducing continuous learning (CL) problems. This continual federated learning (CFL) task presents unique challenges, particularly regarding catastrophic forgetting and non-IID input data. Existing solutions include using a replay buffer to store historical data or leveraging generative adversarial networks. Nevertheless, motivated by recent advancements in the diffusion model for generative tasks, this paper introduces DCFL, a novel framework tailored to address the challenges of CFL in dynamic distributed learning environments. Our approach harnesses the power of the conditional diffusion model to generate synthetic historical data at each local device during communication, effectively mitigating latent shifts in dynamic data distribution inputs. We provide the convergence bound for the proposed CFL framework and demonstrate its promising performance across multiple datasets, showcasing its effectiveness in tackling the complexities of CFL tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。