arXiv:2604.10179math.OCcs.LG2026-04

提出统一分析框架,精准刻画恶意节点下的分布式优化误差边界。

Byzantine-Robust Distributed SGD: A Unified Analysis and Tight Error Bounds

论文配图:Byzantine-Robust Distributed SGD: A Unified Analysis and Tight Error Bounds
图 1 · 摘自论文原文
  • 构建含局部动量的鲁棒分布式SGD统一理论框架
  • 证明随机性与数据异构导致不可消除的误差下限
  • 首次给出紧致上下界,揭示鲁棒性的根本极限

Byzantine 鲁棒分布式优化依赖鲁棒聚合规则来缓解恶意节点的影响。尽管此类规则大量涌现,但缺乏能兼容一般数据异构性的统一收敛分析框架。本文为鲁棒分布式随机梯度下降(SGD)提供了全面的收敛理论,分析了含与不含局部动量的变体。在一般数据异构假设下,建立了非凸光滑目标及满足 Polyak-Łojasiewicz 条件目标的收敛速率。分析表明,虽然随机性和数据异构引入不可避免的误差下限,但局部动量可明确降低由随机性引起的误差分量。此外,我们推导出匹配的下界,证明所获上界是紧致的,刻画了在随机性与数据异构条件下鲁棒性的根本极限。实验结果支持理论发现。

原文摘要 · Abstract (English)

Byzantine-robust distributed optimization relies on robust aggregation rules to mitigate the influence of malicious Byzantine workers. Despite the proliferation of such rules, a unified convergence analysis framework that accommodates general data heterogeneity is lacking. In this work, we provide a thorough convergence theory of Byzantine-robust distributed stochastic gradient descent (SGD), analyzing variants both with and without local momentum. We establish the convergence rates for nonconvex smooth objectives and those satisfying the Polyak-Lojasiewicz condition under a general data heterogeneity assumption. Our analysis reveals that while stochasticity and data heterogeneity introduce unavoidable error floors, local momentum provably reduces the error component induced by stochasticity. Furthermore, we derive matching lower bounds to demonstrate that the upper bounds obtained in our analysis are tight and characterize the fundamental limits of Byzantine resilience under stochasticity and data heterogeneity. Empirical results support our theoretical findings.

分布式优化鲁棒学习异构数据收敛分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。