通过联合剪枝与带宽分配，降低时触发联邦学习通信开销。
TT-Prune: Joint Model Pruning and Resource Allocation for Communication-efficient Time-triggered Federated Learning
- 动态剪枝结合带宽分配优化，提升通信效率。
- 剪枝使通信成本降低40%，模型性能不变。
- 适合资源受限的无线联邦学习场景。
联邦学习（FL）为机器学习提供了新机遇，尤其在解决数据隐私问题方面。与传统事件触发式联邦学习不同，时触发联邦学习（TT-Fed）作为异步与同步联邦学习的统一体，基于固定时间间隔将用户分组。然而，随着用户设备数量增加且无线带宽有限，导致慢节点和通信开销加剧。本文引入自适应模型剪枝到无线TT-Fed系统中，研究联合优化剪枝率与带宽分配的问题，以最小化训练损失并保证最低学习延迟。基于剪枝后的模型梯度l_2范数收敛性分析，构建了在给定延迟阈值下最小化训练损失的联合优化问题。利用Karush-Kuhn-Tucker（KKT）条件推导出带宽和剪枝率的闭式解。仿真结果表明，模型剪枝可使通信成本降低40%，同时保持模型性能不变。
原文摘要 · Abstract (English)
Federated learning (FL) offers new opportunities in machine learning, particularly in addressing data privacy concerns. In contrast to conventional event-based federated learning, time-triggered federated learning (TT-Fed), as a general form of both asynchronous and synchronous FL, clusters users into different tiers based on fixed time intervals. However, the FL network consists of a growing number of user devices with limited wireless bandwidth, consequently magnifying issues such as stragglers and communication overhead. In this paper, we introduce adaptive model pruning to wireless TT-Fed systems and study the problem of jointly optimizing the pruning ratio and bandwidth allocation to minimize the training loss while ensuring minimal learning latency. To answer this question, we perform convergence analysis on the gradient l_2 norm of the TT-Fed model based on model pruning. Based on the obtained convergence upper bound, a joint optimization problem of pruning ratio and wireless bandwidth is formulated to minimize the model training loss under a given delay threshold. Then, we derive closed-form solutions for wireless bandwidth and pruning ratio using Karush-Kuhn-Tucker(KKT) conditions. The simulation results show that model pruning could reduce the communication cost by 40% while maintaining the model performance at the same level.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。