优化物联网分布式训练的调度时间,提出高效算法降低延迟。
Makespan Minimization in Split Learning: From Theory to Practice
- 设计联合分配与调度算法,解决客户端-助手间任务分配问题
- 在同质任务下实现5倍近似比的多项式时间解法
- 针对异构任务提出新启发式方法,实验表现优于已有方案
分裂学习近期成为应对异构物联网设备分布式机器学习的方案,客户端可将部分训练任务卸载至计算能力强的辅助节点。核心挑战在于通过联合设计客户端-助手分配与辅助节点的任务调度来最小化训练时延。我们首先研究每个辅助节点内存容量受限(即只能服务有限客户端数)的同质任务场景,通过复杂性理论证明:即使对高度受限实例,也不存在精确多项式时间算法或近似方案。随后,我们提出一种非平凡的多项式时间5-近似算法作为补充。在此基础上,进一步研究更普遍的异构任务设置(由Tirana等[INFOCOM 2024]提出),其中辅助节点具有内存容量限制,而不同客户端具有可变内存开销。该情况下,我们证明除非P=NP,否则该问题无法存在任何近似因子的多项式时间近似算法。然而,通过改进前述5-近似算法,我们提出一种新型启发式方法,并通过大量实验验证其性能优于已有方法。
原文摘要 · Abstract (English)
Split learning recently emerged as a solution for distributed machine learning with heterogeneous IoT devices, where clients can offload part of their training to computationally-powerful helpers. The core challenge in split learning is to minimize the training time by jointly devising the client-helper assignment and the schedule of tasks at the helpers. We first study the model where each helper has a memory cardinality constraint on how many clients it may be assigned, which represents the case of homogeneous tasks. Through complexity theory, we rule out exact polynomial-time algorithms and approximation schemes even for highly restricted instances of this problem. We complement these negative results with a non-trivial polynomial-time 5-approximation algorithm. Building on this, we then focus on the more general heterogeneous task setting considered by Tirana et al. [INFOCOM 2024], where helpers have memory capacity constraints and clients have variable memory costs. In this case, we prove that, unless P=NP, the problem cannot admit a polynomial-time approximation algorithm for any approximation factor. However, by adapting our aforementioned 5-approximation algorithm, we develop a novel heuristic for the heterogeneous task setting and show that it outperforms heuristics from prior works through extensive experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。