arXiv:2605.02337cs.DCcs.LG2026-05

通过分层训练降低资源消耗,让不同设备高效协作训练模型。

FedPLT: Scalable, Resource-Efficient, and Heterogeneity-Aware Federated Learning via Partial Layer Training

论文配图:FedPLT: Scalable, Resource-Efficient, and Heterogeneity-Aware Federated Learning via Partial Layer Training
图 1 · 摘自论文原文
  • 按设备能力分配模型训练部分,实现个性化参数更新。
  • 减少71%-82%可训练参数,性能媲美全模型训练。
  • 适合资源差异大、通信受限的分布式场景使用。

联邦学习(FL)在保护数据隐私的前提下实现跨分布式系统的协同建模,但面临高通信与计算开销及严重设备异构性挑战。现有方法虽尝试子模型或部分参数训练,却常导致客户端参数分布不一致、全局损失估计不准,引入更高偏差与方差。基于实证分析,本文提出FedPLT(部分层训练联邦学习),一种结构化部分参数训练新范式:根据客户端通信与计算能力动态分配模型部分,训练行为接近全模型训练。结合最优客户端采样,在通信预算约束下显著降低采样方差。大量实验表明,FedPLT在高度异构环境下表现优异,相比全模型训练(如FedAvg)性能相当甚至更优,同时将每客户端可训练参数减少71%-82%,有效缓解慢速客户端问题。

原文摘要 · Abstract (English)

Federated Learning (FL) has gained significant attention in distributed machine learning by enabling collaborative model training across decentralized system while preserving data privacy. Although extensive research has addressed statistical data heterogeneity, FL still faces several challenges, including high communication and computation overheads and severe device heterogeneity, which require further investigation. Prior work has addressed these issues through sub-model training and partial parameter training. However, such methods often suffer from inconsistent parameter distributions across clients, inaccurate global loss estimation, and increased bias and variance. Guided by our empirical analysis, we propose FedPLT (Federated Learning with Partial Layer Training), an innovative and structured partial parameter training approach that exhibits training behavior similar to full model training while assigning client-specific portions of the model according to their communication and computational capabilities. In addition, we evaluate the performance of FedPLT when combined with optimal client sampling under communication constraints. We show that this integration improves FL performance by reducing sampling variance under the same communication budget. Through extensive experiments, we demonstrate that FedPLT achieves performance comparable to, or even surpassing, that of full-model training (i.e., FedAvg), while requiring significantly fewer trainable parameters per client. Moreover, FedPLT outperforms existing methods in highly heterogeneous environments, effectively adapts to client resource constraints, and reduces the number of straggling clients. In particular, FedPLT reduces the number of trainable parameters by 71%-82% while achieving performance on par with full-model training.

联邦学习模型压缩资源效率设备异构

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。