无需中心服务器，高效抗干扰的分布式学习方法
Robust and Efficient Collaborative Learning
- 采用拉取式传染通信机制，降低消息传递开销
- 通信复杂度降至O(n log n)，比传统方法快得多
- 适合大规模分布式系统，尤其抗恶意节点干扰
协同机器学习面临训练时对抗行为的挑战。现有方法要么依赖中心服务器，要么通信成本过高。本文提出鲁棒拉取式传染学习（RPEL），一种新型可扩展协同学习方法，可在存在敌对节点的情况下保证学习鲁棒性。RPEL不依赖任何中心服务器，与传统方法随节点数n呈O(n²)增长不同，其采用基于拉取的传染式通信策略，复杂度仅为O(n log n)。通过从少量随机节点拉取模型参数，显著减少所需消息数量，同时在高概率下保持收敛性。实验表明，RPEL在对抗环境下仍具鲁棒性，精度媲美全连接通信，且在大规模网络中高效可扩展。
原文摘要 · Abstract (English)
Collaborative machine learning is challenged by training-time adversarial behaviors. Existing approaches to tolerate such behaviors either rely on a central server or induce high communication costs. We propose Robust Pull-based Epidemic Learning (RPEL), a novel, scalable collaborative approach to ensure robust learning despite adversaries. RPEL does not rely on any central server and, unlike traditional methods, where communication costs grow in $\mathcal{O}(n^2)$ with the number of nodes $n$, RPEL employs a pull-based epidemic-based communication strategy that scales in $\mathcal{O}(n \log n)$. By pulling model parameters from small random subsets of nodes, RPEL significantly lowers the number of required messages without compromising convergence guarantees, which hold with high probability. Empirical results demonstrate that RPEL maintains robustness in adversarial settings, competes with all-to-all communication accuracy, and scales efficiently across large networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。