为联邦学习设计自动结构化剪枝，显著降低资源消耗。
Automatic Structured Pruning for Efficient Architecture in Federated Learning
- 动态确定剪枝边界，无需依赖本地数据
- 参数量减少89%，计算量降低90%，精度损失小
- 适合移动端部署，通信开销降5倍、推理提速50%
在联邦学习中，训练在客户端设备上进行，通常受限于计算资源和存储容量。为此，我们提出一种专为联邦学习系统设计的自动剪枝方案，提升客户端计算效率并降低通信成本。由于难以获取本地数据，传统剪枝超参数调优面临挑战，因此我们引入一种动态确定剪枝边界的自动剪枝范式。此外，采用针对无稀疏计算硬件支持的移动设备优化的结构化剪枝算法。实验表明，该方法在FEMNIST和CelebFaces数据集上达到与现有方法相当的精度，参数量减少89%，计算量（FLOPS）降低90%，通信开销最多降低5倍，安卓设备上推理时间缩短一半。
原文摘要 · Abstract (English)
In Federated Learning (FL), training is conducted on client devices, typically with limited computational resources and storage capacity. To address these constraints, we propose an automatic pruning scheme tailored for FL systems. Our solution improves computation efficiency on client devices, while minimizing communication costs. One of the challenges of tuning pruning hyper-parameters in FL systems is the restricted access to local data. Thus, we introduce an automatic pruning paradigm that dynamically determines pruning boundaries. Additionally, we utilized a structured pruning algorithm optimized for mobile devices that lack hardware support for sparse computations. Experimental results demonstrate the effectiveness of our approach, achieving accuracy comparable to existing methods. Our method notably reduces the number of parameters by 89% and FLOPS by 90%, with minimal impact on the accuracy of the FEMNIST and CelebFaces datasets. Furthermore, our pruning method decreases communication overhead by up to 5x and halves inference time when deployed on Android devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。