arXiv:2608.15153cs.CRcs.LG2026-08

自适应梯度裁剪与噪声注入提升隐私联邦学习的效率与精度

An Adaptive Gradient Clipping and Noise Injection Mechanism for Differentially Private Federated Learning

论文配图:An Adaptive Gradient Clipping and Noise Injection Mechanism for Differentially Private Federated Learning
图 1 · 摘自论文原文
  • 每轮按层动态调整梯度裁剪阈值,基于样本梯度范数中位数
  • 训练效率提升19.21%,测试损失降低98.74%,精度显著提高
  • 在更小隐私预算(ε=0.1)下达到高精度,适合高隐私场景

差分隐私联邦学习需在隐私保护、模型精度与训练效率间权衡。静态梯度裁剪采用固定阈值,易导致过度裁剪或添加过多噪声。本文提出DDP-SA-adaptive,一种支持安全聚合的自适应梯度裁剪与噪声注入机制。每轮通信中,各客户端根据每层样本梯度范数的中位数,为各层独立确定裁剪阈值,使阈值随梯度分布变化而动态调整,并据此校准拉普拉斯噪声。在联邦回归任务上评估显示,相比静态基线DDP-SA,该方法减少6.81%通信轮次、总训练时间降低19.21%、每轮训练时间减少13.33%,提升训练效率;测试损失降低98.74%,测试R²提升3.41%;实现R²=0.99仅需ε≈0.1,而静态方法需ε=0.4,显著增强隐私保护。结果表明,轮次-层级自适应可优化差分隐私联邦学习的隐私-精度-效率平衡。

原文摘要 · Abstract (English)

Differentially private federated learning must balance privacy protection against model accuracy and training efficiency. Static gradient clipping applies a fixed threshold throughout training and across model layers, which can cause excessive clipping when the threshold is too small or unnecessarily large noise when it is too large. This paper presents DDP-SA-adaptive, an adaptive gradient clipping and noise adding mechanism for differentially private federated learning with secure aggregation. At each communication round, every client determines a separate clipping threshold for each model layer from the median of its per-sample gradient norms. The resulting layer-wise thresholds adapt to the evolving gradient distributions and calibrate the Laplace noise added before the updates are encoded and secret-shared among intermediate aggregation servers. We evaluate the proposed mechanism on a federated regression task in terms of efficiency, accuracy, privacy, convergence, clipping norm, and noise magnitude. Compared with the static DDP-SA baseline, DDP-SA-adaptive reduces the number of communication rounds by 6.81%, total training time by 19.21%, and average per-round training time by 13.33%, leading to improved training efficiency. It also reduces test loss by 98.74% and increases test R2 by 3.41%, leading to improved model accuracy. To attain R2 = 0.99, the adaptive mechanism operates with a privacy budget of approximately epsilon = 0.1, compared with epsilon = 0.4 for static DDP-SA, thus providing stronger privacy protection and achieving stronger privacy guarantees. These results demonstrate that round-wise, layer-wise adaptation can improve the privacy-accuracy-efficiency trade-off of differentially private federated learning.

隐私计算联邦学习自适应

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。