提出PubSub-VFL框架,显著提升异构环境下的双方联邦学习效率。
PubSub-VFL: Towards Efficient Two-Party Split Learning in Heterogeneous Environments via Publisher/Subscriber Architecture
- 采用发布/订阅架构解耦通信,实现分层异步训练。
- 相比基线提速2~7倍,资源利用率最高达91.07%。
- 适合资源不均的多方协作场景,兼顾隐私与性能。
随着数字经济快速发展,组织间数据协作已成为主流商业模式,但隐私问题使直接共享数据不可行。为此,两方分割学习(即垂直联邦学习,VFL)成为安全协同学习的有力方案。然而,其同步依赖设计导致训练延迟高,参与者间的计算与数据异构性进一步制约效率。为此,本文提出PubSub-VFL,一种基于发布/订阅架构的新型双方协作学习范式,具备高计算效率。该方法利用发布/订阅的解耦能力与参数服务器的数据并行特性,设计分层异步机制,降低训练延迟并提升系统效率。此外,针对资源与数据异构引发的训练不平衡问题,基于参与方系统特征形式化优化问题,实现隐私保护下的最优超参选择。理论分析表明,PubSub-VFL具有稳定收敛性,兼容差分隐私等安全协议。在五个基准数据集上的大量实验验证其有效性:相比先进基线,训练速度提升2~7倍,精度不受损,计算资源利用率最高达91.07%。
原文摘要 · Abstract (English)
With the rapid advancement of the digital economy, data collaboration between organizations has become a well-established business model, driving the growth of various industries. However, privacy concerns make direct data sharing impractical. To address this, Two-Party Split Learning (a.k.a. Vertical Federated Learning (VFL)) has emerged as a promising solution for secure collaborative learning. Despite its advantages, this architecture still suffers from low computational resource utilization and training efficiency. Specifically, its synchronous dependency design increases training latency, while resource and data heterogeneity among participants further hinder efficient computation. To overcome these challenges, we propose PubSub-VFL, a novel VFL paradigm with a Publisher/Subscriber architecture optimized for two-party collaborative learning with high computational efficiency. PubSub-VFL leverages the decoupling capabilities of the Pub/Sub architecture and the data parallelism of the parameter server architecture to design a hierarchical asynchronous mechanism, reducing training latency and improving system efficiency. Additionally, to mitigate the training imbalance caused by resource and data heterogeneity, we formalize an optimization problem based on participants' system profiles, enabling the selection of optimal hyperparameters while preserving privacy. We conduct a theoretical analysis to demonstrate that PubSub-VFL achieves stable convergence and is compatible with security protocols such as differential privacy. Extensive case studies on five benchmark datasets further validate its effectiveness, showing that, compared to state-of-the-art baselines, PubSub-VFL not only accelerates training by $2 \sim 7\times$ without compromising accuracy, but also achieves a computational resource utilization rate of up to 91.07%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。