arXiv:2412.08282cs.LGcs.AI2024-12被引 1

提出平滑近似方法提升联邦对抗学习泛化能力,解决非光滑损失难题。

How Does the Smoothness Approximation Method Facilitate Generalization for Federated Adversarial Learning?

  • 用三种平滑近似法分析算法稳定性,评估泛化性能
  • 随机平滑法(RSA)最有效降低泛化误差
  • 数据异构时推荐使用松弛型FAL缓解性能下降

联邦对抗学习(FAL)是一种抵御联邦学习中对抗攻击的鲁棒框架。尽管已有研究提出高效算法,但多关注收敛性而忽视泛化能力。由于对抗损失函数非光滑,泛化分析更具挑战性。本文通过引入平滑近似,构建算法稳定性度量,评估两种主流FAL算法——基础型FAL(VFAL)与松弛型FAL(SFAL)在三种平滑近似方法下的泛化表现:代理平滑近似(SSA)、随机平滑近似(RSA)与过参数平滑近似(OPSA)。理论分析表明,合理选择平滑方法可有效降低泛化误差;其中RSA表现最优。在高度数据异构场景下,建议采用SFAL以缓解异构导致的泛化性能退化。研究为设计更高效的FAL算法提供依据,如新指标与动态聚合规则以应对异构性。

原文摘要 · Abstract (English)

Federated Adversarial Learning (FAL) is a robust framework for resisting adversarial attacks on federated learning. Although some FAL studies have developed efficient algorithms, they primarily focus on convergence performance and overlook generalization. Generalization is crucial for evaluating algorithm performance on unseen data. However, generalization analysis is more challenging due to non-smooth adversarial loss functions. A common approach to addressing this issue is to leverage smoothness approximation. In this paper, we develop algorithm stability measures to evaluate the generalization performance of two popular FAL algorithms: \textit{Vanilla FAL (VFAL)} and {\it Slack FAL (SFAL)}, using three different smooth approximation methods: 1) \textit{Surrogate Smoothness Approximation (SSA)}, (2) \textit{Randomized Smoothness Approximation (RSA)}, and (3) \textit{Over-Parameterized Smoothness Approximation (OPSA)}. Based on our in-depth analysis, we answer the question of how to properly set the smoothness approximation method to mitigate generalization error in FAL. Moreover, we identify RSA as the most effective method for reducing generalization error. In highly data-heterogeneous scenarios, we also recommend employing SFAL to mitigate the deterioration of generalization performance caused by heterogeneity. Based on our theoretical results, we provide insights to help develop more efficient FAL algorithms, such as designing new metrics and dynamic aggregation rules to mitigate heterogeneity.

联邦学习对抗学习泛化分析平滑近似

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。