基于联邦学习实现跨云大模型隐私保护协同训练
Research on Large Language Model Cross-Cloud Privacy Protection and Collaborative Training based on Federated Learning
- 融合密码学与动态聚合，构建跨云安全协作框架
- 混合聚合方案提升模型准确率与稳定性，降低数据泄露风险
- 适合关注跨云隐私计算的科研与工程人员
大语言模型快速发展与云计算普及，使跨云部署与训练中的隐私保护与数据安全成为关键挑战。本文提出一种新框架，基于联邦学习实现分布式云间的隐私保护协同训练。机制融合前沿密码学原语、动态模型聚合技术与跨云数据协调方案，显著提升传统联邦学习在安全性、效率与可扩展性方面的表现。进一步提出混合聚合方案，有效缓解数据泄露威胁并优化模型更新聚合，大幅增强模型效果与稳定性。实验表明，所提方法在训练效率、隐私保护能力与模型准确率上均优于传统联邦学习。
原文摘要 · Abstract (English)
The fast development of large language models (LLMs) and popularization of cloud computing have led to increasing concerns on privacy safeguarding and data security of cross-cloud model deployment and training as the key challenges. We present a new framework for addressing these issues along with enabling privacy preserving collaboration on training between distributed clouds based on federated learning. Our mechanism encompasses cutting-edge cryptographic primitives, dynamic model aggregation techniques, and cross-cloud data harmonization solutions to enhance security, efficiency, and scalability to the traditional federated learning paradigm. Furthermore, we proposed a hybrid aggregation scheme to mitigate the threat of Data Leakage and to optimize the aggregation of model updates, thus achieving substantial enhancement on the model effectiveness and stability. Experimental results demonstrate that the training efficiency, privacy protection, and model accuracy of the proposed model compare favorably to those of the traditional federated learning method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。