用元学习动态调整丢弃率，提升联邦学习在非独立同分布数据下的泛化能力。
Federated Learning via Meta-Variational Dropout
- 通过共享超网络预测客户端专属丢弃率，实现个性化模型训练。
- 在稀疏非独立同分布数据下，分类准确率与不确定性校准均显著优于基线。
- 压缩本地模型参数，降低过拟合风险和通信开销，适合资源受限场景。
联邦学习（FL）旨在从分布式客户端中训练全局推理模型，因其提升数据隐私的特性而广受欢迎。然而，传统联邦学习常面临模型过拟合及局部模型发散的问题，主要源于客户端数据有限且非独立同分布（non-IID）。为此，我们提出一种新颖的贝叶斯元学习方法——元变分丢弃（MetaVD）。MetaVD通过共享超网络学习客户端相关的丢弃率，实现联邦学习算法在有限非独立同分布数据下的有效模型个性化。我们还从后验适应视角和贝叶斯联邦学习的后验聚合视角，揭示了条件丢弃后验的意义。在多种稀疏且非独立同分布的联邦学习数据集上进行了广泛实验。结果表明，MetaVD在分类准确率和不确定性校准方面表现优异，尤其在分布外（OOD）客户端上优势明显。此外，该方法压缩了每个客户端所需的本地模型参数，缓解了过拟合问题并降低了通信成本。代码已开源：https://github.com/insujeon/MetaVD。
原文摘要 · Abstract (English)
Federated Learning (FL) aims to train a global inference model from remotely distributed clients, gaining popularity due to its benefit of improving data privacy. However, traditional FL often faces challenges in practical applications, including model overfitting and divergent local models due to limited and non-IID data among clients. To address these issues, we introduce a novel Bayesian meta-learning approach called meta-variational dropout (MetaVD). MetaVD learns to predict client-dependent dropout rates via a shared hypernetwork, enabling effective model personalization of FL algorithms in limited non-IID data settings. We also emphasize the posterior adaptation view of meta-learning and the posterior aggregation view of Bayesian FL via the conditional dropout posterior. We conducted extensive experiments on various sparse and non-IID FL datasets. MetaVD demonstrated excellent classification accuracy and uncertainty calibration performance, especially for out-of-distribution (OOD) clients. MetaVD compresses the local model parameters needed for each client, mitigating model overfitting and reducing communication costs. Code is available at https://github.com/insujeon/MetaVD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。