提出新型几何中位数算法，提升抗拜占庭攻击的分布式学习稳定性
Approximate Agreement Algorithms for Byzantine Collaborative Learning
- 用超盒算法实现几何中位数聚合，增强鲁棒性
- 在非独立同分布数据下，可容忍符号翻转攻击
- 适合存在恶意节点的分布式机器学习场景
在拜占庭协同学习中，n个客户端通过点对点网络协作训练模型，不共享原始数据，仅交换并聚合随机梯度估计。恶意客户端可能阻止其他客户端获得相同的梯度集合。因此，聚合步骤需结合高效（近似）共识子程序以保证训练收敛。本文研究几何中位数聚合规则在拜占庭协同学习中的应用。我们发现现有方法无法为共识子程序提供收敛性或梯度质量的理论保证。为此，我们提出一种超盒算法用于几何中位数聚合，满足理论要求。我们在中心化与去中心化设置下，针对非独立同分布数据进行了实际评估，结果表明基于几何中位数的方法在面对符号翻转攻击时，比已有均值方法更具鲁棒性。
原文摘要 · Abstract (English)
In Byzantine collaborative learning, $n$ clients in a peer-to-peer network collectively learn a model without sharing their data by exchanging and aggregating stochastic gradient estimates. Byzantine clients can prevent others from collecting identical sets of gradient estimates. The aggregation step thus needs to be combined with an efficient (approximate) agreement subroutine to ensure convergence of the training process. In this work, we study the geometric median aggregation rule for Byzantine collaborative learning. We show that known approaches do not provide theoretical guarantees on convergence or gradient quality in the agreement subroutine. To satisfy these theoretical guarantees, we present a hyperbox algorithm for geometric median aggregation. We practically evaluate our algorithm in both centralized and decentralized settings under Byzantine attacks on non-i.i.d. data. We show that our geometric median-based approaches can tolerate sign-flip attacks better than known mean-based approaches from the literature.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。