提出最优鲁棒性度量，证明MultiKrum比Krum更抗恶意攻击。
Byzantine Machine Learning: MultiKrum and an optimal notion of robustness
- 引入最优鲁棒性系数κ⋆，更精准衡量对抗环境下均值估计的可靠性
- 首次证明MultiKrum具有理论鲁棒性，其鲁棒性在真实场景下优于Krum
- 为分布式学习中的抗拜占庭攻击聚合规则提供新分析框架，适合安全敏感场景
聚合规则是分布式（或联邦）学习中应对敌手攻击的核心机制，对应于拜占庭威胁模型。从鲁棒均值估计的角度看，这些规则也具有重要的数学意义。尽管Krum已被广泛研究并具备正式的鲁棒性与收敛性保证，但其自然扩展MultiKrum在实践中表现更优，此前却缺乏理论支持。本文首次证明MultiKrum是鲁棒聚合规则，并界定了其鲁棒性系数。为此，我们引入κ⋆——一种最优鲁棒性系数，相较于以往度量能更紧密地刻画对抗环境下的均值估计精度。我们构建了MultiKrum鲁棒性系数的上下界，同时改进了已有对Krum鲁棒性系数的最佳界。结果表明，MultiKrum的界从不劣于Krum，在实际场景中更优。通过实验验证了下界质量。
原文摘要 · Abstract (English)
Aggregation rules are the cornerstone of distributed (or federated) learning in the presence of adversaries, under the so-called Byzantine threat model. They are also interesting mathematical objects from the point of view of robust mean estimation. The Krum aggregation rule has been extensively studied, and endowed with formal robustness and convergence guarantees. Yet, MultiKrum, a natural extension of Krum, is often preferred in practice for its superior empirical performance, even though no theoretical guarantees were available until now. In this work, we provide the first proof that MultiKrum is a robust aggregation rule, and bound its robustness coefficient. To do so, we introduce $κ^\star$, the optimal *robustness coefficient* of an aggregation rule, which quantifies the accuracy of mean estimation in the presence of adversaries in a tighter manner compared with previously adopted notions of robustness. We then construct an upper and a lower bound on MultiKrum's robustness coefficient. As a by-product, we also improve on the best-known bounds on Krum's robustness coefficient. We show that MultiKrum's bounds are never worse than Krum's, and better in realistic regimes. We illustrate this analysis by an experimental investigation on the quality of the lower bound.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。