设计激励机制防止联邦学习中客户端篡改梯度。
Incentivizing Truthful Collaboration in Heterogeneous Federated Learning
- 提出博弈模型，通过支付规则让客户端无法通过修改梯度获利。
- 理论证明支付规则可抑制篡改，且保证模型收敛速度。
- 实验验证在图像与文本任务中对多种聚合策略均有效。
联邦学习（FL）是一种分布式协作学习方法，多个客户端通过共享梯度更新而非原始数据共同学习。然而，FL易受客户端恶意更新的影响。本文研究数据异质性对客户端篡改更新动机的影响。首先，构建了客户端可通过修改梯度获利的异构协作场景，并发现此类篡改会导致模型性能下降。为此，我们建立一个博弈模型，其中客户端可能谎报梯度以“引导”服务器模型向自身有利方向发展。我们设计了一种支付规则，在FedSGD协议下可严格抑制发送修改后梯度的动机。推导出客户端支付和全局模型收敛率的显式边界，从而分析异质性、支付与收敛之间的权衡。最后，在计算机视觉与自然语言处理三个任务上，对FedSGD、基于中位数聚合的FedSGD及FedAvg协议进行了实验评估。结果表明，该方案在所有情况下均有效抑制了更新篡改。
原文摘要 · Abstract (English)
Federated learning (FL) is a distributed collaborative learning method, where multiple clients learn together by sharing gradient updates instead of raw data. However, it is well-known that FL is vulnerable to manipulated updates from clients. In this work we study the impact of data heterogeneity on clients' incentives to manipulate their updates. First, we present heterogeneous collaborative learning scenarios where a client can modify their updates to be better off, and show that these manipulations can lead to diminishing model performance. To prevent such modifications, we formulate a game in which clients may misreport their gradient updates in order to "steer" the server model to their advantage. We develop a payment rule that provably disincentivizes sending modified updates under the FedSGD protocol. We derive explicit bounds on the clients' payments and the convergence rate of the global model, which allows us to study the trade-off between heterogeneity, payments and convergence. Finally, we provide an experimental evaluation of the effectiveness of our payment rule in the FedSGD, median-based aggregation FedSGD and FedAvg protocols on three tasks in computer vision and natural language processing. In all cases we find that our scheme successfully disincentivizes modifications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。