提出抗拜占庭攻击的优化方法，可在复杂函数下稳定收敛。
Byzantine-Robust Optimization under $(L_0, L_1)$-Smoothness
- 用动量归一化与近邻混合提升鲁棒性
- 收敛速度达O(K^{-1/4})，受攻击偏差可控
- 适合异构数据和多种攻击场景
针对具有状态相关梯度Lipschitz常数的$(L_0,L_1)$-光滑函数，在拜占庭攻击下的分布式优化问题，本文提出 Byz-NSGDM 方法。该算法结合动量归一化与近邻混合增强的鲁棒聚合策略，同时应对函数平滑性和恶意节点干扰。理论证明其收敛速率为$O(K^{-1/4})$，最终误差受鲁棒系数与梯度异质性影响。在异构MNIST分类、合成$(L_0,L_1)$-光滑优化及小规模GPT字符级语言建模任务中验证了对多种拜占庭攻击的有效性。消融实验表明，该方法在广泛动量与学习率设置下仍保持鲁棒性。
原文摘要 · Abstract (English)
We consider distributed optimization under Byzantine attacks in the presence of $(L_0,L_1)$-smoothness, a generalization of standard $L$-smoothness that captures functions with state-dependent gradient Lipschitz constants. We propose Byz-NSGDM, a normalized stochastic gradient descent method with momentum that achieves robustness against Byzantine workers while maintaining convergence guarantees. Our algorithm combines momentum normalization with Byzantine-robust aggregation enhanced by Nearest Neighbor Mixing (NNM) to handle both the challenges posed by $(L_0,L_1)$-smoothness and Byzantine adversaries. We prove that Byz-NSGDM achieves a convergence rate of $O(K^{-1/4})$ up to a Byzantine bias floor proportional to the robustness coefficient and gradient heterogeneity. Experimental validation on heterogeneous MNIST classification, synthetic $(L_0,L_1)$-smooth optimization, and character-level language modeling with a small GPT model demonstrates the effectiveness of our approach against various Byzantine attack strategies. An ablation study further shows that Byz-NSGDM is robust across a wide range of momentum and learning rate choices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。