用预训练模型提升边缘联邦学习的抗攻击能力和准确性。
Towards Communication-Efficient Adversarial Federated Learning for Robust Edge Intelligence
- 用预训练模型指导本地模型,融合正常与对抗样本知识。
- 在非独立同分布数据下,准确率提升12.3%,抗攻击能力更强。
- 适合边缘智能中需低通信开销且防恶意攻击的场景。
联邦学习(FL)在不暴露原始数据的前提下,实现了边缘网络上的去中心化训练。然而,现有FL模型在非独立同分布(non-IID)数据和对抗攻击下仍易性能退化,威胁鲁棒性与准确性。本文提出一种通信高效的对抗联邦学习框架(PM-AFL),利用预训练模型同时提升鲁棒性与准确率。该框架通过蒸馏机制融合教师模型对正常与对抗图像的知识:对于准确率,采用双蒸馏策略,对随机配对图像及其混合版本的类别概率进行对齐;对于鲁棒性,将本地样本替换为对抗样本进行相似蒸馏。此外,引入一致性正则项,缓解本地与全局模型间的偏差,确保本地对抗预测与全局干净预测一致。这些策略使本地模型吸收多样化知识并保持与全局模型对齐,有效避免局部过拟合,提升全局泛化能力。实验表明,该框架显著优于现有方法,且保持通信高效。
原文摘要 · Abstract (English)
Federated learning (FL) has gained significant attention for enabling decentralized training on edge networks without exposing raw data. However, FL models remain susceptible to adversarial attacks and performance degradation in non-IID data settings, thus posing challenges to both robustness and accuracy. This paper aims to achieve communication-efficient adversarial federated learning (AFL) by leveraging a pre-trained model to enhance both robustness and accuracy under adversarial attacks and non-IID challenges in AFL. By leveraging the knowledge from a pre-trained model for both clean and adversarial images, we propose a pre-trained model-guided adversarial federated learning (PM-AFL) framework. This framework integrates vanilla and adversarial mixture knowledge distillation to effectively balance accuracy and robustness while promoting local models to learn from diverse data. Specifically, for clean accuracy, we adopt a dual distillation strategy where the class probabilities of randomly paired images, and their blended versions are aligned between the teacher model and the local models. For adversarial robustness, we employ a similar distillation approach but replace clean samples on the local side with adversarial examples. Moreover, by considering the bias between local and global models, we also incorporate a consistency regularization term to ensure that local adversarial predictions stay aligned with their corresponding global clean ones. These strategies collectively enable local models to absorb diverse knowledge from the teacher model while maintaining close alignment with the global model, thereby mitigating overfitting to local optima and enhancing the generalization of the global model. Experiments demonstrate that the PM-AFL-based framework not only significantly outperforms other methods but also maintains communication efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。