用正交投影压缩通信量并防信息泄露,提升分割学习隐私与效率。
LightSplit: Practical Privacy-Preserving Split Learning via Orthogonal Projections

- 在分割层使用固定正交随机投影,降低传输维度
- 32倍降维下仍保持95%以上基准准确率
- 无需额外训练组件,适合边缘设备部署
分割学习(SL)通过将神经网络分置于客户端与中心服务器实现协作训练,但切分层的高维激活导致通信开销大且易受重建攻击。现有方法多仅关注效率或隐私,依赖稀疏化、量化或加噪等附加机制。本文提出LightSplit,于切分层引入轻量级固定正交随机投影,依据香农信息论构建信息瓶颈,抑制实例相关信号与可被利用的样本级特征。通过传输低维投影而非原始激活,服务器在不改变架构的前提下处理升维表示,保持与现有SL结构兼容。由于投影不可逆,客户端丢弃部分原始信息,降低重构可能性。实验在多种独立同分布(IID)与非独立同分布(non-IID)场景下验证,不同投影维度与客户端规模下,方法在最大32倍传输维度缩减时仍保留超95%基准准确率,训练过程稳定。
原文摘要 · Abstract (English)
Split learning (SL) enables collaborative training by partitioning a neural network across clients and a central server, but the cut-layer interface introduces a key challenge: high-dimensional activations incur substantial communication overhead while exposing representations vulnerable to reconstruction attacks. Existing approaches typically address efficiency or privacy in isolation, relying on additional mechanisms such as sparsification, quantization, or noise injection. We propose LightSplit, which limits information exposure and reduces communication overhead by applying a lightweight fixed orthogonal random projection at the cut layer. Based on Shannon's information theory, this projection acts as an information bottleneck that restricts instance-specific information and suppresses exploitable per-sample signals. By transmitting low-dimensional projections instead of raw activations, the server operates on lifted representations without requiring architectural modifications, ensuring compatibility with existing SL architectures. By avoiding additional trainable components on the client, the method remains lightweight and suitable for edge devices while preserving end-to-end differentiability via exact gradient propagation. As the projection is non-invertible, part of the original representation is irreversibly discarded at the client, LightSplit reduces the information available for reconstruction and limits information exposure. We extensively evaluate LightSplit on state-of-the-art benchmarks in both IID and non-IID settings across varying projection dimensions and client scales. Our results show that the method retains more than 95% of the baseline accuracy at up to 32x reduction in transmitted dimensionality while maintaining stable training dynamics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。