ZORRO让分片学习更安全,防止恶意客户端注入后门。
ZORRO: Zero-Knowledge Robustness and Privacy for Split Learning (Full Version)
- 用零知识证明验证客户端正确执行防御算法
- 攻击成功率降至6%以下,客户端100万参数仅增10秒开销
- 适合隐私敏感、资源受限的分布式模型训练场景
分片学习(SL)是一种分布式学习方法,使资源受限的客户端可通过将大部分网络层卸载到中心服务器来协作训练深度神经网络(DNN),同时保持输入输出层在本地。该设计利用服务器算力且不共享数据,适用于处理敏感数据的资源受限环境。然而,其分布式特性允许恶意客户端通过发送被污染的中间梯度,向共享的DNN注入后门。现有防御措施多聚焦于服务器端保护,且常带来额外开销。我们提出ZORRO,一种私密、可验证、鲁棒的分片学习防御方案。通过创新设计并应用交互式零知识证明(ZKPs),客户端证明其正确执行了本地部署的防御算法,从而生成计算完整性证明,表明本地训练部分无害。结合模型分区的频域表示,ZORRO可在不受信任环境中对本地训练模型进行深入检查,确保每个客户端向后续客户端传递的是无害检查点。在涵盖不同模型架构、多种攻击策略及数据场景的广泛评估中,我们证明了ZORRO的有效性:攻击成功率低于6%,即使客户端存储100万个参数,额外开销也小于10秒。
原文摘要 · Abstract (English)
Split Learning (SL) is a distributed learning approach that enables resource-constrained clients to collaboratively train deep neural networks (DNNs) by offloading most layers to a central server while keeping in- and output layers on the client-side. This setup enables SL to leverage server computation capacities without sharing data, making it highly effective in resource-constrained environments dealing with sensitive data. However, the distributed nature enables malicious clients to manipulate the training process. By sending poisoned intermediate gradients, they can inject backdoors into the shared DNN. Existing defenses are limited by often focusing on server-side protection and introducing additional overhead for the server. A significant challenge for client-side defenses is enforcing malicious clients to correctly execute the defense algorithm. We present ZORRO, a private, verifiable, and robust SL defense scheme. Through our novel design and application of interactive zero-knowledge proofs (ZKPs), clients prove their correct execution of a client-located defense algorithm, resulting in proofs of computational integrity attesting to the benign nature of locally trained DNN portions. Leveraging the frequency representation of model partitions enables ZORRO to conduct an in-depth inspection of the locally trained models in an untrusted environment, ensuring that each client forwards a benign checkpoint to its succeeding client. In our extensive evaluation, covering different model architectures as well as various attack strategies and data scenarios, we show ZORRO's effectiveness, as it reduces the attack success rate to less than 6\% while causing even for models storing \numprint{1000000} parameters on the client-side an overhead of less than 10 seconds.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。