提出新型垂直联邦学习框架,通信量减少99%且保持模型性能。
The Sherpa.ai Blind Vertical Federated Learning Paradigm to Minimize the Number of Communications
- 将节点更新与服务器解耦,大幅减少通信次数
- 相比标准垂直联邦学习,通信量降低约99%
- 适合医疗、金融等敏感领域使用
联邦学习(FL)可在多个参与方(节点)间实现协作式去中心化训练,同时保护原始数据隐私。目前主要有两种范式:水平联邦学习(HFL),各节点共享相同特征空间但样本不同;垂直联邦学习(VFL),各节点对同一样本持有互补特征。尽管HFL应用广泛,但在需要互补特征的场景中仍需采用VFL。然而VFL存在显著缺陷:训练过程中通信量巨大,损害隐私与安全,导致高能耗,甚至因通信开销过高使模型训练不可行。本文提出Sherpa.ai盲垂直联邦学习(SBVFL),一种增强隐私与安全性的分布式训练新范式。通过将绝大多数节点更新与服务器解耦,显著降低节点-服务器通信量。实验表明,SBVFL相比标准VFL通信量减少约99%,同时保持准确性和鲁棒性。因此,该方法使在医疗、金融、制造、航天、网络安全及国防等敏感领域实现可实践的隐私保护型垂直联邦学习成为可能。
原文摘要 · Abstract (English)
Federated Learning (FL) enables collaborative decentralized training across multiple parties (nodes) while keeping raw data private. There are two main paradigms in FL: Horizontal FL (HFL), where all participant nodes share the same feature space but hold different samples, and Vertical FL (VFL), where participants hold complementary features for the same samples. While HFL is widely adopted, VFL is employed in domains where nodes hold complementary features about the same samples. Still, VFL presents a significant limitation: the vast number of communications required during training. This compromises privacy and security, and can lead to high energy consumption, and in some cases, make model training unfeasible due to the high number of communications. In this paper, we introduce Sherpa.ai Blind Vertical Federated Learning (SBVFL), a novel paradigm that leverages a distributed training mechanism enhanced for privacy and security. Decoupling the vast majority of node updates from the server dramatically reduces node-server communication. Experiments show that SBVFL reduces communication by ~99% compared to standard VFL while maintaining accuracy and robustness. Therefore, SBVFL enables practical, privacy-preserving VFL across sensitive domains, including healthcare, finance, manufacturing, aerospace, cybersecurity, and the defense industry.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。