arXiv:2605.28335cs.LG2026-05

通过降维加速联邦学习抗攻击聚合,显著提升效率且不损失性能。

Dimensionality Reduction for Robust Federated Learning: A Theoretical Analysis and Convergence Guarantee

  • 用稀疏随机投影将梯度压缩到低维子空间,快速计算可靠性权重。
  • 计算复杂度降至理论最优的 $\mathcal{O}(Mp)$,训练速度提升数个量级。
  • 适用于大规模模型联邦学习,尤其适合对效率敏感的工业场景。

联邦学习(FL)允许多个客户端在不共享原始数据的情况下协作训练模型,但极易受到拜占庭攻击。现有鲁棒方法虽能抵御攻击,但在高维梯度聚合时带来巨大计算开销,且随模型规模增长而加剧。为此,本文提出投影降维(PDR)框架,适用于基于向量距离的鲁棒聚合器,通过稀疏随机投影将梯度压缩至极小子空间,高效计算可靠性权重。该方法将服务器计算复杂度降至最优的 $\mathcal{O}(Mp)$,与读取梯度所需的理论下界一致。在标准联邦学习假设下,建立了收敛性保证:对于非凸函数,收敛率为 $\mathcal{O}(1/\sqrt{T})$;对于强凸函数,为 $\mathcal{O}(1/T)$,其中 $T$ 为迭代次数。关键证明显示,这种大幅加速几乎无代价,仅使固有的拜占庭误差底限增加一个可调有界因子 $\frac{1+ε}{1-ε}$。基准数据集上的实验表明,将 PDR 与现有聚合器结合,可在保持优异收敛性能的同时实现数个量级的时间效率提升。

原文摘要 · Abstract (English)

Federated Learning (FL) enables multiple clients to collaboratively train models without sharing raw data, but it is highly vulnerable to Byzantine attacks. Existing robust approaches can neutralize these threats but incur substantial computational overhead during high-dimensional gradient aggregation, an overhead that scales poorly with model size and increasingly dominates the training cost as modern models grow larger. To address this computational bottleneck, we propose Projected Dimensionality Reduction (PDR), a universal acceleration framework for vector-level distance-based robust aggregators, which performs robust aggregation by compressing gradients into a drastically smaller subspace via sparse random projection to efficiently compute reliability weights. This approach reduces the server computational complexity to an optimal $ \mathcal{O}(Mp) $, where $ M $ is the number of clients and $ p $ is the model dimension, matching the theoretical lower bound required merely to read the gradients. We establish convergence guarantees under standard FL assumptions in prior Byzantine-robust FL analyses. By leveraging the Subspace Embedding Theorem, we show that PDR achieves optimal convergence rates of $ \mathcal{O}(1/\sqrt{T}) $ for non-convex functions and $ \mathcal{O}(1/T) $ for strongly convex functions, where $ T $ denotes the number of iterations. Crucially, we mathematically demonstrate that this massive acceleration comes almost for free, merely inflating the inherent Byzantine error floor by a bounded, tunable factor of $ \frac{1+ε}{1-ε} $. Experimental results on benchmark datasets confirm that integrating PDR with existing aggregators yields orders of magnitude speedups in time efficiency while maintaining highly competitive convergence performance.

联邦学习降维鲁棒性加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。