提出自适应步长机制，加速隐私保护联邦学习收敛
Accelerating Differentially Private Federated Learning via Adaptive Extrapolation
- 根据本地更新差异自适应调整全局步长
- 理论证明可加速收敛，实测优于现有方法
- 无需额外超参数或客户端计算开销
联邦学习（FL）使多方可在不共享原始数据的情况下协作训练模型，但仍面临隐私攻击风险。引入差分隐私（DP）是应对该问题的有力方案。目前最流行的DP-FL算法是DP-FedAvg，但在客户端数据异构情况下收敛速度慢。现有加速方法通常需额外超参数或增加客户端计算成本，前者调参代价高且易引发隐私泄露，后者不适用于资源受限的客户端。为此，本文提出DP-FedEXP，通过分析本地更新的多样性自适应选择全局步长，无需额外超参数或客户端计算开销。理论证明其可加速DP-FedAvg收敛，实验结果表明其性能优于针对DP-FL设计的现有方法。
原文摘要 · Abstract (English)
The federated learning (FL) framework enables multiple clients to collaboratively train machine learning models without sharing their raw data, but it remains vulnerable to privacy attacks. One promising approach is to incorporate differential privacy (DP)-a formal notion of privacy-into the FL framework. DP-FedAvg is one of the most popular algorithms for DP-FL, but it is known to suffer from the slow convergence in the presence of heterogeneity among clients' data. Most of the existing methods to accelerate DP-FL require 1) additional hyperparameters or 2) additional computational cost for clients, which is not desirable since 1) hyperparameter tuning is computationally expensive and data-dependent choice of hyperparameters raises the risk of privacy leakage, and 2) clients are often resource-constrained. To address this issue, we propose DP-FedEXP, which adaptively selects the global step size based on the diversity of the local updates without requiring any additional hyperparameters or client computational cost. We show that DP-FedEXP provably accelerates the convergence of DP-FedAvg and it empirically outperforms existing methods tailored for DP-FL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。