arXiv:2410.18075cs.LGcs.IT2024-10被引 1

解决联邦学习中模型更新导致数据分布漂移的优化难题。

ProFL: Performative Robust Optimal Federated Learning

  • 提出新算法在噪声数据下实现最优联邦学习
  • 在非凸目标下保证收敛到最优解
  • 适合真实场景中分布漂移严重的联邦学习任务

表现性预测框架捕捉了模型部署过程中因自身影响导致的数据分布变化。当训练模型被使用时,数据生成过程会引发模型演化,从而偏离原始数据分布。这种由模型引发的分布偏移在联邦学习中的影响在实际应用中日益显著。近期方法将表现性预测扩展至联邦学习,使模型收敛至表现性稳定点,但该点可能远离表现性最优点。先前中心化设置的研究表明,在模型诱导分布偏移下可达到表现性最优点,但需假设表现性风险为凸且训练数据无噪声,这些条件在真实联邦学习系统中常不成立。本文克服上述局限,提出表现性鲁棒最优联邦学习(ProFL),可在含噪声和污染数据下找到联邦学习中的表现性最优点。我们在多个数据集上进行了广泛实验,结果表明该方法优于现有最先进方法。在Polyak-Łojasiewicz条件下完成收敛分析,适用于非凸目标。

原文摘要 · Abstract (English)

Performative prediction is a framework that captures distribution shifts that occur during the training of machine learning models due to their deployment. As the trained model is used, data generation causes the model to evolve, leading to deviations from the original data distribution. The impact of such model-induced distribution shifts in federated learning is increasingly likely to transpire in real-life use cases. A recently proposed approach extends performative prediction to federated learning with the resulting model converging to a performative stable point, which may be far from the performative optimal point. Earlier research in centralized settings has shown that the performative optimal point can be achieved under model-induced distribution shifts, but these approaches require the performative risk to be convex and the training data to be noiseless, assumptions often violated in realistic federated learning systems. This paper overcomes all of these shortcomings and proposes Performative Robust Optimal Federated Learning, an algorithm that finds performative optimal points in federated learning from noisy and contaminated data. We present the convergence analysis under the Polyak-Lojasiewicz condition, which applies to non-convex objectives. Extensive experiments on multiple datasets demonstrate the advantage of Robust Optimal Federated Learning over the state-of-the-art.

联邦学习分布偏移优化算法非凸

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。