arXiv:2411.05260cs.CRcs.AI2024-11被引 10

用量化剪枝压缩加密通信,让联邦学习更安全高效

QuanCrypt-FL: Quantized Homomorphic Encryption with Pruning for Secure Federated Learning

  • 结合低比特量化与剪枝,降低加密计算开销
  • 加密速度比BatchCrypt快9倍,训练时间减少3倍
  • 适合对隐私和效率都有要求的分布式学习场景

联邦学习通过客户端协作训练模型而不交换私有数据,提升了数据隐私。然而在训练和推理阶段仍易受梯度反演、成员推断等攻击。同态加密可保护模型更新,但带来巨大通信开销。为此,我们提出QuanCrypt-FL,将低比特量化与剪枝技术结合,显著降低训练时的计算成本,同时提升抗攻击能力。进一步引入基于均值的裁剪机制,缓解量化溢出或误差问题。该框架在MNIST、CIFAR-10和CIFAR-100上验证,相比现有方法表现更优:在不同客户端数量下,准确率接近原始联邦学习(Vanilla-FL),加密速度最快达BatchCrypt的9倍,解密速度快16倍,推理速度提升1.5倍,训练时间最多缩短3倍。

原文摘要 · Abstract (English)

Federated Learning has emerged as a leading approach for decentralized machine learning, enabling multiple clients to collaboratively train a shared model without exchanging private data. While FL enhances data privacy, it remains vulnerable to inference attacks, such as gradient inversion and membership inference, during both training and inference phases. Homomorphic Encryption provides a promising solution by encrypting model updates to protect against such attacks, but it introduces substantial communication overhead, slowing down training and increasing computational costs. To address these challenges, we propose QuanCrypt-FL, a novel algorithm that combines low-bit quantization and pruning techniques to enhance protection against attacks while significantly reducing computational costs during training. Further, we propose and implement mean-based clipping to mitigate quantization overflow or errors. By integrating these methods, QuanCrypt-FL creates a communication-efficient FL framework that ensures privacy protection with minimal impact on model accuracy, thereby improving both computational efficiency and attack resilience. We validate our approach on MNIST, CIFAR-10, and CIFAR-100 datasets, demonstrating superior performance compared to state-of-the-art methods. QuanCrypt-FL consistently outperforms existing method and matches Vanilla-FL in terms of accuracy across varying client. Further, QuanCrypt-FL achieves up to 9x faster encryption, 16x faster decryption, and 1.5x faster inference compared to BatchCrypt, with training time reduced by up to 3x.

联邦学习加密量化剪枝

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。