HSL框架让分布式学习更高效，通信量少一半还能保持更高精度。
Hubs and Spokes Learning: Efficient and Scalable Collaborative Machine Learning
- 采用中心-边缘双层通信结构，避免单点故障。
- 用400条连接达1000条的性能，资源受限系统更适用。
- 节点间共识更强，训练轮数更少，适合大规模部署。
我们提出一种新型协同机器学习范式——中心-边缘学习（Hubs and Spokes Learning, HSL），融合联邦学习（FL）与去中心化学习（P2PL）的优势。HSL采用双层通信结构，避免了FL的单点故障问题，并在相同通信预算下优于当前最先进的去中心化框架Epidemic Learning Local（ELL）。在同等通信预算下，HSL性能超越ELL；而在显著更低的通信预算下，其性能可与ELL相当。例如，在CIFAR-10数据集上，100个节点（边缘）仅用400条通信边时，即可达到ELL使用1000条边的测试准确率。此外，HSL在信息混合后实现更强的节点共识，从而在更少训练轮次中提升性能。通过严格的理论分析和广泛的实验验证，证明了HSL在大规模协同学习中的实用性。
原文摘要 · Abstract (English)
We introduce the Hubs and Spokes Learning (HSL) framework, a novel paradigm for collaborative machine learning that combines the strengths of Federated Learning (FL) and Decentralized Learning (P2PL). HSL employs a two-tier communication structure that avoids the single point of failure inherent in FL and outperforms the state-of-the-art P2PL framework, Epidemic Learning Local (ELL). At equal communication budgets (total edges), HSL achieves higher performance than ELL, while at significantly lower communication budgets, it can match ELL's performance. For instance, with only 400 edges, HSL reaches the same test accuracy that ELL achieves with 1000 edges for 100 peers (spokes) on CIFAR-10, demonstrating its suitability for resource-constrained systems. HSL also achieves stronger consensus among nodes after mixing, resulting in improved performance with fewer training rounds. We substantiate these claims through rigorous theoretical analyses and extensive experimental results, showcasing HSL's practicality for large-scale collaborative learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。