arXiv:2506.09438cs.LGcs.DC2025-06被引 2

分析去中心化学习在数据异质下的泛化误差,揭示攻击与异质性的影响。

Generalization Error Analysis for Attack-Free and Byzantine-Resilient Decentralized Learning with Data Heterogeneity

  • 基于异质数据和温和假设,分析无攻击与抗拜占庭攻击的泛化误差
  • 发现恶意攻击对泛化误差影响显著,且与样本量无关
  • 适用于研究分布式学习鲁棒性与泛化性能的研究者

去中心化学习通过地理分散的智能体联合训练模型,在信号与信息处理领域备受关注。尽管优化误差已有广泛研究,泛化误差仍相对未被深入探讨。泛化误差反映模型在未见数据上的可扩展性,对实际应用性能至关重要。本文针对具有数据异质性的去中心化学习,在无攻击和抗拜占庭攻击场景下,进行细粒度的泛化误差分析,突破了以往仅考虑同质数据或依赖严苛有界随机梯度假设的局限。研究揭示了数据异质性、模型初始化及随机梯度噪声对泛化误差的影响,此前未受充分关注。同时发现,恶意节点发起的拜占庭攻击会显著恶化泛化误差,其负面影响与数据异质性密切相关,但不随样本规模变化。在凸与非凸任务上进行数值实验,验证了理论结果的有效性。

原文摘要 · Abstract (English)

Decentralized learning, which facilitates joint model training across geographically scattered agents, has gained significant attention in the field of signal and information processing in recent years. While the optimization errors of decentralized learning algorithms have been extensively studied, their generalization errors remain relatively under-explored. As the generalization errors reflect the scalability of trained models on unseen data and are crucial in determining the performance of trained models in real-world applications, understanding the generalization errors of decentralized learning is of paramount importance. In this paper, we present fine-grained generalization error analysis for both attack-free and Byzantine-resilient decentralized learning with heterogeneous data as well as under mild assumptions, in contrast to prior studies that consider homogeneous data and/or rely on a stringent bounded stochastic gradient assumption. Our results shed light on the impact of data heterogeneity, model initialization and stochastic gradient noise -- factors that have not been closely investigated before -- on the generalization error of decentralized learning. We also reveal that Byzantine attacks performed by malicious agents largely affect the generalization error, and their negative impact is inherently linked to the data heterogeneity while remaining independent on the sample size. Numerical experiments on both convex and non-convex tasks are conducted to validate our theoretical findings.

去中心化学习泛化误差拜占庭鲁棒性数据异质

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。