Byzantine故障比数据投毒更损害模型泛化能力，首次揭示两者在泛化上的根本差距。
Tight Stability Bounds for Robust Distributed Learning: Byzantine Failures Hurt Generalization More than Data Poisoning
- 通过算法稳定性分析，对比了两种攻击对泛化的影响机制
- 当f个节点失效时，Byzantine攻击的泛化误差上界更差
- 适用于关注分布式学习鲁棒性与泛化性能的研究者
鲁棒分布式学习算法旨在应对异常工作节点带来的干扰。这类异常通常建模为两类： extit{Byzantine故障}（通信可被任意篡改）和 extit{数据投毒}（仅限本地训练数据被污染）。尽管已有研究显示两者在优化性能上具有相似保障，但一个关键问题仍未解决： extit{这两种威胁模型对泛化性能有何影响？} 本文首次揭示两者在泛化保障上的根本差异：Byzantine故障导致的泛化误差率严格劣于数据投毒情形。这一结论基于对鲁棒分布式学习算法的紧致算法稳定性分析。具体而言，在n个节点中有f个发生故障的情况下，我们证明：(i) 在数据投毒模型下，鲁棒分布式学习算法具有统一算法稳定性；(ii) 而在Byzantine故障下，其稳定性界更差，进而导致更差的泛化性能。该结果强调了在设计鲁棒系统时需区分攻击类型的重要性。
原文摘要 · Abstract (English)
Robust distributed learning algorithms aim to maintain reliable performance despite the presence of misbehaving workers. Such misbehaviors are commonly modeled as \textit{Byzantine failures}, allowing arbitrarily corrupted communication, or as \textit{data poisoning}, a weaker form of corruption restricted to local training data. While prior work shows similar optimization guarantees for both models, an important question remains: \textit{How do these threat models impact generalization?} We show, for the first time, a fundamental gap in generalization guarantees between the two threat models: Byzantine failures yield strictly worse rates than those achievable under data poisoning. Our findings are based upon a tight algorithmic stability analysis of robust distributed learning. Specifically, with $f$ out of $n$ workers misbehaving, we prove that: \textit{(i)} under data poisoning, the uniform algorithmic stability of a robust distributed learning algorithm
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。