通过分层聚合与余弦距离提升联邦学习抗拜占庭攻击能力
Improving $(α, f)$-Byzantine Resilience in Federated Learning via layerwise aggregation and cosine distance
- 分层处理模型参数，用余弦距离筛选可信更新
- 在多种攻击下最高提升16%准确率，优于传统方法
- 适合高维模型防御，兼顾效率与鲁棒性
人工智能快速发展引发社会对数据隐私的关切，联邦学习（FL）通过不共享数据实现协同训练，成为解决隐私问题的潜在方案。然而，FL系统易受拜占庭攻击，恶意节点会提交污染的模型更新。尽管已有鲁棒聚合算法被广泛采用，但在高维参数空间中性能显著下降，导致模型表现不佳。本文提出分层余弦聚合（Layerwise Cosine Aggregation），旨在提升此类算法在高维场景下的鲁棒性，同时保持计算高效。理论分析表明，该方法相比原始鲁棒聚合器具有更强的抗干扰能力。在多个图像分类数据集上，面对不同数据分布和攻击模式的实证测试均显示，该方法可实现最高16%的准确率提升，验证了其有效性。
原文摘要 · Abstract (English)
The rapid development of artificial intelligence systems has amplified societal concerns regarding their usage, necessitating regulatory frameworks that encompass data privacy. Federated Learning (FL) is posed as potential solution to data privacy challenges in distributed machine learning by enabling collaborative model training {without data sharing}. However, FL systems remain vulnerable to Byzantine attacks, where malicious nodes contribute corrupted model updates. While Byzantine Resilient operators have emerged as a widely adopted robust aggregation algorithm to mitigate these attacks, its efficacy diminishes significantly in high-dimensional parameter spaces, sometimes leading to poor performing models. This paper introduces Layerwise Cosine Aggregation, a novel aggregation scheme designed to enhance robustness of these rules in such high-dimensional settings while preserving computational efficiency. A theoretical analysis is presented, demonstrating the superior robustness of the proposed Layerwise Cosine Aggregation compared to original robust aggregation operators. Empirical evaluation across diverse image classification datasets, under varying data distributions and Byzantine attack scenarios, consistently demonstrates the improved performance of Layerwise Cosine Aggregation, achieving up to a 16% increase in model accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。