提出自适应引导方法，让异构联邦学习更协同高效
Adaptive Guidance for Local Training in Heterogeneous Federated Learning
- 用可学习的引导信号动态调整本地训练目标
- 在14种异构模型上均超越现有7种先进方法
- 仅需一阶梯度，理论保证收敛性，适合实际部署
模型异构性是异构联邦学习（HtFL）面临的主要挑战。当客户端采用不同模型架构时，直接参数聚合不可行，现有方法通常在本地目标外增加额外目标以促进协作，但常导致目标不匹配。为此，本文提出联邦学习引导（FedL2G），一种以联邦方式自适应学习本地训练引导的方法，确保新增目标与各客户端原始目标对齐。该方法具备理论保障，仅需模型参数的一阶导数，实现非凸优化下 O(1/T) 的收敛速率。我们在两种数据异构和六种模型异构设置下进行广泛实验，使用14种异构模型架构（如CNNs和ViTs）。结果表明，FedL2G显著优于七种当前最优方法。
原文摘要 · Abstract (English)
Model heterogeneity poses a significant challenge in Heterogeneous Federated Learning (HtFL). In scenarios with diverse model architectures, directly aggregating model parameters is impractical, leading HtFL methods to incorporate an extra objective alongside the original local objective on each client to facilitate collaboration. However, this often results in a mismatch between the extra and local objectives. To resolve this, we propose Federated Learning-to-Guide (FedL2G), a method that adaptively learns to guide local training in a federated manner, ensuring the added objective aligns with each client's original goal. With theoretical guarantees, FedL2G utilizes only first-order derivatives w.r.t. model parameters, achieving a non-convex convergence rate of O(1/T). We conduct extensive experiments across two data heterogeneity and six model heterogeneity settings, using 14 heterogeneous model architectures (e.g., CNNs and ViTs). The results show that FedL2G significantly outperforms seven state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。