通过自适应剪枝压缩中间数据，降低分布式训练通信开销
Exploiting Adaptive Channel Pruning for Communication-Efficient Split Learning
- 根据标签感知重要性动态剪掉不关键通道
- 在保持精度前提下减少训练轮次和通信量
- 适合资源受限设备的高效联邦学习场景
分割学习（SL）将大部分训练任务转移至服务器，减轻客户端计算负担。然而，中间特征表示（即“压碎数据”）的传输带来了显著通信开销，尤其在大量客户端参与时更为突出。为此，我们提出一种自适应通道剪枝辅助的分割学习（ACP-SL）方案。在ACP-SL中，设计了标签感知通道重要性评分（LCIS）模块生成通道重要性得分，区分关键与非关键通道；基于该得分，构建自适应通道剪枝（ACP）模块，剪除次要通道，从而压缩对应压碎数据并降低通信开销。实验结果表明，ACP-SL在测试精度上持续优于基准方案，并在更少训练轮次内达到目标精度，有效减少通信开销。
原文摘要 · Abstract (English)
Split learning (SL) transfers most of the training workload to the server, which alleviates computational burden on client devices. However, the transmission of intermediate feature representations, referred to as smashed data, incurs significant communication overhead, particularly when a large number of client devices are involved. To address this challenge, we propose an adaptive channel pruning-aided SL (ACP-SL) scheme. In ACP-SL, a label-aware channel importance scoring (LCIS) module is designed to generate channel importance scores, distinguishing important channels from less important ones. Based on these scores, an adaptive channel pruning (ACP) module is developed to prune less important channels, thereby compressing the corresponding smashed data and reducing the communication overhead. Experimental results show that ACP-SL consistently outperforms benchmark schemes in test accuracy. Furthermore, it reaches a target test accuracy in fewer training rounds, thereby reducing communication overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。