arXiv:2502.19170cs.LG2025-02被引 3

证明了signSGD在强敌攻击下仍能收敛,突破了此前理论局限。

On the Byzantine Fault Tolerance of signSGD with Majority Vote

  • 构建全知敌手模型,分析其最优破坏策略
  • 推导出错误聚合的显式概率上界,给出收敛速率
  • 适用于对抗性强、合谋的分布式学习场景

在分布式学习中,基于符号的压缩算法如signSGD with majority vote 提供了轻量级替代方案,并具备近乎免费的容错能力。然而,现有研究仅证明其对较弱敌手(非全知、无法合谋)具有容错性。本文填补这一空白,首次分析全知且可合谋敌手的最强攻击情形。通过定义全知框架并分析无限制攻击者下的最大破坏策略,发现符号压缩的过滤效应将攻击空间约束至最优破坏方向。由此推导出不依赖未知常数的错误聚合显式概率上界,建立含拜占庭攻击者时signSGD with majority vote的收敛边界与精确收敛速率。实验在MNIST分布式环境中验证了不同强度敌手下的有效性。

原文摘要 · Abstract (English)

In distributed learning, sign-based compression algorithms such as signSGD with majority vote provide a lightweight alternative to SGD with an additional advantage: fault tolerance (almost) for free. However, for signSGD with majority vote, this fault tolerance has been shown to cover only the case of weaker adversaries, i.e., ones that are not omniscient or cannot collude to base their attack on common knowledge and strategy. In this work, we close this gap and provide new insights into how signSGD with majority vote can be resilient against omniscient and colluding adversaries, which craft an attack after communicating with other adversaries, thus having better information to perform the most damaging attack based on a common optimal strategy. Our core contribution is in providing a proof that begins by defining the omniscience framework and the strongest possible damage against signSGD with majority vote without imposing any restrictions on the attacker. Thanks to the filtering effect of the sign-based method, we upper-bound the space of attacks to the optimal strategy for maximizing damage by an attacker. Hence, we derive an explicit probabilistic bound in terms of incorrect aggregation without resorting to unknown constants, providing a convergence bound on signSGD with majority vote in the presence of Byzantine attackers, along with a precise convergence rate. Our findings are supported by experiments on the MNIST dataset in a distributed learning environment with adversaries of varying strength.

分布式学习拜占庭容错signSGD

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。