arXiv:2502.06915cs.DCcs.LG2025-02被引 1

提出可个性化更新的联邦元学习框架,显著加速训练并提升异构数据下模型性能。

Analytic Personalized Federated Meta-Learning

  • 通过分层最小二乘法实现DNN的单步全局更新,避免梯度计算
  • 在异构数据上测试准确率比基线高4%~8%,部分场景达最新水平
  • 适合需要快速适配本地数据的边缘智能应用

分析型联邦学习(AFL)是一种无需梯度的增强型联邦学习范式,通过闭式最小二乘解在单步内更新全局模型以加速训练。然而,当客户端数据分布异构时,所得全局模型性能下降。元学习常用于生成个性化本地模型以缓解此问题。但将元学习与AFL结合面临双重挑战:其一,传统AFL框架不支持深度神经网络(DNN)训练,影响元学习对复杂联邦任务的快速适应能力;其二,现有元学习方法依赖梯度信息,而AFL不涉及梯度。为此,我们提出新框架FedACnnL,通过将每层训练建模为分布式最小二乘问题,实现层间协同的DNN训练。针对第二重挑战,进一步提出分析式个性化联邦元学习框架pFedACnnL,通过解析求解局部目标函数,桥接全局模型与个体数据分布之间的差距。理论证明,FedACnnL在DNN训练中所需时间比传统联邦学习框架减少83%~99%。实验表明,pFedACnnL在测试准确率上相较基线提高4%~8%,在大多数凸与非凸设置下达到当前最优性能。

原文摘要 · Abstract (English)

Analytic Federated Learning (AFL) is an enhanced gradient-free federated learning (FL) paradigm designed to accelerate training by updating the global model in a single step with closed-form least-square (LS) solutions. However, the obtained global model suffers performance degradation across clients with heterogeneous data distribution. Meta-learning is a common approach to tackle this problem by delivering personalized local models for individual clients. Yet, integrating meta-learning with AFL presents significant challenges: First, conventional AFL frameworks cannot support deep neural network (DNN) training which can influence the fast adaption capability of meta-learning for complex FL tasks. Second, the existing meta-learning method requires gradient information, which is not involved in AFL. To overcome the first challenge, we propose an AFL framework, namely FedACnnL, in which a layer-wise DNN collaborative training method is designed by modeling the training of each layer as a distributed LS problem. For the second challenge, we further propose an analytic personalized federated meta-learning framework, namely pFedACnnL. It generates a personalized model for each client by analytically solving a local objective which bridges the gap between the global model and the individual data distribution. FedACnnL is theoretically proven to require significantly shorter training time than the conventional FL frameworks on DNN training while the reduction ratio is $83\%\sim99\%$ in the experiment. Meanwhile, pFedACnnL excels at test accuracy with the vanilla FedACnnL by $4\%\sim8\%$ and it achieves state-of-the-art (SOTA) model performance in most cases of convex and non-convex settings compared with previous SOTA frameworks.

联邦学习元学习DNN加速个性化建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。