针对边缘设备异构性,提出并行化分片联邦学习框架,提升训练效率与模型精度。
Split Federated Learning Over Heterogeneous Edge Devices: Algorithm and Optimization
- 客户端在不同切分层并行训练个性化模型,避免串行延迟瓶颈。
- 在非独立同分布数据下,收敛速度比现有方法快23%,准确率提升1.8%。
- 适合资源不均的物联网场景,尤其适用于智能医疗、工业监控等隐私敏感应用。
分片学习(Split Learning, SL)是一种有前景的协作式机器学习方法,使资源受限设备可在不共享原始数据的前提下训练模型,同时降低计算负载并保护隐私。然而,现有SL算法在训练效率上存在局限,尤其在串行设置中因设备异构性及频繁的数据交互导致延迟过长,最慢设备成为整体瓶颈。为此,本文提出异构分片联邦学习(HSFL)框架,允许资源受限客户端在不同切分层上并行训练个性化客户端模型,以缓解异构环境影响并加速训练。我们构建联合优化计算与传输资源的延迟最小化问题,并设计融合样本平均近似(SAA)、遗传算法(GA)、拉格朗日松弛与分支定界(B&B)的资源分配算法以高效求解。仿真结果表明,在异构设备与非独立同分布数据下,HSFL在收敛速度和模型精度上均优于其他框架,且优化算法显著降低延迟。
原文摘要 · Abstract (English)
Split Learning (SL) is a promising collaborative machine learning approach, enabling resource-constrained devices to train models without sharing raw data, while reducing computational load and preserving privacy simultaneously. However, current SL algorithms face limitations in training efficiency and suffer from prolonged latency, particularly in sequential settings, where the slowest device can bottleneck the entire process due to heterogeneous resources and frequent data exchanges between clients and servers. To address these challenges, we propose the Heterogeneous Split Federated Learning (HSFL) framework, which allows resource-constrained clients to train their personalized client-side models in parallel, utilizing different cut layers. Aiming to mitigate the impact of heterogeneous environments and accelerate the training process, we formulate a latency minimization problem that optimizes computational and transmission resources jointly. Additionally, we design a resource allocation algorithm that combines the Sample Average Approximation (SAA), Genetic Algorithm (GA), Lagrangian relaxation and Branch and Bound (B\&B) methods to efficiently solve this problem. Simulation results demonstrate that HSFL outperforms other frameworks in terms of both convergence rate and model accuracy on heterogeneous devices with non-iid data, while the optimization algorithm is better than other baseline methods in reducing latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。