解决跨超算中心训练中的队列延迟问题，提升联邦学习效率。
FedQueue: Queue-Aware Federated Learning for Cross-Facility HPC Training
- 根据队列延迟动态调整本地计算量，避免资源浪费。
- 通过截止机制缓冲晚到设备，将更新过时程度控制在可接受范围。
- 适合异构设备多、任务调度不稳定的跨设施联邦学习场景。
跨多个超算中心的联邦学习面临批次调度器带来的随机准入延迟，显著影响实际训练时间。同步联邦学习受慢节点严重拖累，而异步方法在队列高峰时累积过时更新。本文提出 FedQueue，一种感知队列延迟的联邦学习协议，直接将调度延迟纳入训练与聚合过程：(i) 在线预测各设施的队列延迟，用于规划本地计算；(ii) 采用基于截止的准入机制，缓冲延迟到达的设备以限制过时程度；(iii) 实施过时感知聚合，稳定异构本地负载。我们证明了在过时程度有界条件下，非凸目标的收敛速率为 $/mathcal{O}(1/ ext{sqrt}{R})$，且在队列预测误差有限时，准入控制可高概率保证过时程度有界。真实跨设施部署显示，相比基线算法提升20.5%。受控队列仿真表明，在高队列方差和非独立同分布划分下，达到目标精度所需时间最多减少60%。
原文摘要 · Abstract (English)
Federated learning (FL) across multiple HPC facilities faces stochastic admission delays from batch schedulers that dominate wall-clock time. Synchronous FL suffers from severe stragglers, while asynchronous FL accumulates stale updates when queues spike. We propose FedQueue, a queue-aware FL protocol that incorporates scheduler delays directly into training and aggregation, which (i) predicts per-facility queue delays online to budget local work, (ii) applies cutoff-based admission that buffers late arrivals to bound staleness, and (iii) performs staleness-aware aggregation to stabilize heterogeneous local workloads. We prove the convergence for non-convex objectives at rate $\mathcal{O}(1/\sqrt{R})$ under bounded staleness, and show that the admission controls yield bounded staleness with high probability under queue-prediction error. Real-world cross-facility deployment of FedQueue shows 20.5% improvement over baseline algorithms. Controlled queue simulations demonstrate robust improvement over the baselines; in particular, up to 60% reduction in time to reach a target accuracy level under high queue variance and non-IID partitions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。