提出可支持任意轮微调的元联邦学习框架，提升个性化模型适应能力。
A Generalized Meta Federated Learning Framework with Theoretical Convergence Guarantees
- 通过最小化任意轮微调后的平均损失，构建更通用的元联邦学习框架
- 理论证明算法收敛性，实验证明在真实数据集上精度更高、收敛更快
- 适合数据异构性强的场景，尤其适用于需多轮微调的个性化任务
元联邦学习（Meta FL）是一种个性化联邦学习方法，多个代理在不交换原始数据样本的情况下协同训练一个初始共享模型。该初始模型应能通过一次或少数几次微调快速适配到各代理的本地数据，从而提升模型个性化效果。传统元FL方法仅最小化单步微调后各代理的平均损失。然而在实际中，尤其当各代理数据分布高度异构时，可能需要多轮微调才能有效适配全局模型。为此，本文提出一种广义元联邦学习框架，通过最小化任意数量 $ν$ 轮微调后的平均损失来优化全局模型。针对该框架，我们设计了一种改进的联邦平均（FedAvg）算法，并对精确与近似两种情形下的收敛速度及元损失函数行为进行了全面的理论分析。实验结果表明，所提方法在真实数据集上相比传统方法具有更高的准确率和更快的收敛速度。
原文摘要 · Abstract (English)
Meta federated learning (FL) is a personalized variant of FL, where multiple agents collaborate on training an initial shared model without exchanging raw data samples. The initial model should be trained in a way that current or new agents can easily adapt it to their local datasets after one or a few fine-tuning steps, thus improving the model personalization. Conventional meta FL approaches minimize the average loss of agents on the local models obtained after one step of fine-tuning. In practice, agents may need to apply several fine-tuning steps to adapt the global model to their local data, especially under highly heterogeneous data distributions across agents. To this end, we present a generalized framework for the meta FL by minimizing the average loss of agents on their local model after any arbitrary number $ν$ of fine-tuning steps. For this generalized framework, we present a variant of the well-known federated averaging (FedAvg) algorithm and conduct a comprehensive theoretical convergence analysis to characterize the convergence speed as well as behavior of the meta loss functions in both the exact and approximated cases. Our experiments on real-world datasets demonstrate superior accuracy and faster convergence for the proposed scheme compared to conventional approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。