arXiv:2503.21159cs.LGcs.AI2025-03被引 2

动态调整梯度裁剪,平衡隐私与模型精度

Multi-Objective Optimization for Privacy-Utility Balance in Differentially Private Federated Learning

  • 用多目标优化动态调节梯度裁剪阈值
  • 在相同隐私约束下,准确率优于固定裁剪方法
  • 适合关注隐私保护的联邦学习研究者

联邦学习(FL)可在不共享原始数据的情况下实现跨客户端协同建模,是隐私保护机器学习的有前景方案。然而,在联邦学习中保障差分隐私(DP)面临模型效用与隐私保护之间的权衡挑战。通常在聚合前对梯度进行裁剪以限制隐私损失,但选择最优裁剪范数极具挑战:过高的值会损害隐私,过严的裁剪则降低模型性能。本文提出一种自适应裁剪机制,基于多目标优化框架动态调整裁剪范数。通过将隐私与效用纳入优化目标,该方法有效平衡了隐私保护与模型准确性。我们理论分析了方法的收敛性,并在MNIST、Fashion-MNIST和CIFAR-10数据集上进行了广泛实验。结果表明,自适应裁剪始终优于固定裁剪基线,在相同隐私约束下实现了更高的准确率。本工作凸显了动态裁剪策略在提升差分隐私联邦学习中隐私-效用权衡潜力。

原文摘要 · Abstract (English)

Federated learning (FL) enables collaborative model training across distributed clients without sharing raw data, making it a promising approach for privacy-preserving machine learning. However, ensuring differential privacy (DP) in FL presents challenges due to the trade-off between model utility and privacy protection. Clipping gradients before aggregation is a common strategy to limit privacy loss, but selecting an optimal clipping norm is non-trivial, as excessively high values compromise privacy, while overly restrictive clipping degrades model performance. In this work, we propose an adaptive clipping mechanism that dynamically adjusts the clipping norm using a multi-objective optimization framework. By integrating privacy and utility considerations into the optimization objective, our approach balances privacy preservation with model accuracy. We theoretically analyze the convergence properties of our method and demonstrate its effectiveness through extensive experiments on MNIST, Fashion-MNIST, and CIFAR-10 datasets. Our results show that adaptive clipping consistently outperforms fixed-clipping baselines, achieving improved accuracy under the same privacy constraints. This work highlights the potential of dynamic clipping strategies to enhance privacy-utility trade-offs in differentially private federated learning.

联邦学习差分隐私多目标优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。