arXiv:2601.19261cs.LGcs.AI2026-01被引 1

通过辅助分类器实现客户端与服务器模型解耦训练,大幅降低通信和内存开销。

Decoupled Split Learning via Auxiliary Loss

  • 客户端加小辅助分类器提供本地误差信号,与服务器分开展示训练
  • 通信量减少50%,峰值内存使用降低至原方案的42%
  • 适合资源受限设备部署,尤其适用于隐私保护场景

分割学习是一种分布式训练范式,将神经网络在客户端与服务器间分割,使数据保留在客户端,仅共享中间激活值。传统分割学习依赖客户端与服务器间端到端反向传播,导致每轮迭代需交换前向激活值和反向梯度,通信开销大且内存占用高。本文提出一种超越反向传播的训练方法:客户端与服务器分别基于本地损失信号半独立训练。具体地,客户端在分割点添加小型辅助分类器以提供本地误差信号,服务器则基于客户端传输的激活值使用真实损失函数训练。该解耦设计无需传递反向梯度,使通信成本减半,同时因双方仅需存储自身反向传播所需的本地激活值,显著降低内存开销。我们在CIFAR-10和CIFAR-100上评估该方法,结果表明:性能与传统基于反向传播的分割学习相当;通信量减少约50%,峰值内存使用最高下降58%。

原文摘要 · Abstract (English)

Split learning is a distributed training paradigm where a neural network is partitioned between clients and a server, which allows data to remain at the client while only intermediate activations are shared. Traditional split learning relies on end-to-end backpropagation across the client-server split point. This incurs a large communication overhead (i.e., forward activations and backward gradients need to be exchanged every iteration) and significant memory use (for storing activations and gradients). In this paper, we develop a beyond-backpropagation training method for split learning. In this approach, the client and server train their model partitions semi-independently, using local loss signals instead of propagated gradients. In particular, the client's network is augmented with a small auxiliary classifier at the split point to provide a local error signal, while the server trains on the client's transmitted activations using the true loss function. This decoupling removes the need to send backward gradients, which cuts communication costs roughly in half and also reduces memory overhead (as each side only stores local activations for its own backward pass). We evaluate our approach on CIFAR-10 and CIFAR-100. Our experiments show two key results. First, the proposed approach achieves performance on par with standard split learning that uses backpropagation. Second, it significantly reduces communication (of transmitting activations/gradient) by 50% and peak memory usage by up to 58%.

分割学习通信优化内存节省隐私计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。