arXiv:2510.07922cs.LGcs.DC2025-10被引 2

提出SketchGuard,用压缩映射筛选恶意节点,通信量与模型大小无关。

SketchGuard: Scaling Byzantine-Robust Decentralized Federated Learning via Sketch-Based Screening

  • 先在压缩的计数草图域筛选邻居,通过再取全模型,降低通信开销。
  • 在6种攻击下性能媲美顶尖方法,1100万参数语言模型上有效。
  • 新协议防适应性攻击,适合大规模去中心化联邦学习场景。

去中心化联邦学习(DFL)中的拜占庭鲁棒性可抵御恶意客户端。现有防御依赖相似性过滤,需交换完整模型向量,通信量随模型维度增长,扩展性差。本文提出SketchGuard,将筛选与聚合解耦:客户端在紧凑的计数草图域中筛选邻居,仅对通过筛选的邻居获取完整模型。我们发现直接实现存在安全漏洞:因草图是固定公开的线性映射,自适应攻击者可将任意大扰动隐藏于其零空间,使污染模型同时通过草图筛选和重采样验证。我们证明此漏洞,并提出commit-then-sketch协议——在模型提交后才生成草图种子,恢复不可知性设置,使计数草图能严格保留筛选决策。进一步建立了强凸与非凸场景下的收敛性,明确依赖网络连通性与数据异构性。实验表明,经保护的SketchGuard在六类攻击下达到顶尖精度,仅小幅阈值膨胀,覆盖多种网络拓扑与异构设置,并在1100万参数语言模型的去中心化微调任务中有效,同时将每邻居筛选通信量降至与模型维度无关。

原文摘要 · Abstract (English)

Byzantine-robust decentralized federated learning (DFL) protects peer-to-peer training from malicious clients. The dominant defenses rely on similarity-based filtering, in which each client exchanges full model vectors with every neighbor before any filtering decision; this communication grows with the model dimension and scales poorly as models grow. We propose SketchGuard, which decouples screening from aggregation: clients screen neighbors in a compact Count Sketch domain and fetch full models only from those that pass the screen. We show this idea is insecure when implemented naively. Because the sketch is a fixed, publicly known linear map, an adaptive adversary can hide an arbitrarily large perturbation in its null space, so the poisoned model passes both the sketch-domain filter and the re-sketch verification. We prove this vulnerability and close it with commit-then-sketch, a one-message protocol that draws the sketch seed only after models are committed, restoring the oblivious setting in which Count Sketch provably preserves screening decisions. We then establish convergence in strongly convex and non-convex settings, with explicit dependence on network connectivity and data heterogeneity. Empirically, secured SketchGuard matches state-of-the-art full-precision robustness, up to a small threshold inflation, across six attacks including the adaptive null-space attack, a range of network topologies and heterogeneity settings, and a decentralized fine-tuning task on an 11-million-parameter language model, while reducing per-neighbor screening communication to a size independent of the model dimension.

联邦学习拜占庭鲁棒通信压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。