提出新型量化方法,兼顾隐私保护与异构设备协同训练效率
Privacy-Preserving Quantized Federated Learning with Diverse Precision
- 设计随机量化器实现差分隐私与最小量化误差
- 在异构精度下仍保持模型聚合准确率,优于传统方法
- 适合存在数据隐私需求的多设备分布式学习场景
联邦学习(FL)作为一种分布式机器学习范式,使多个本地设备可在不共享原始数据的情况下协作训练全局模型。然而,其仍受限于:(i)本地模型更新传输至融合中心时缺乏保护引发的隐私风险;(ii)参与设备间模型量化分辨率异构导致的学习效用下降。现有工作通常仅解决其中一问题,因在同时应对隐私风险与量化异构下维持学习效用极具挑战。本文旨在提升隐私保护联邦学习中异构量化精度设备参与下的学习效用。提出一种新型随机量化器(SQ),可同时实现差分隐私(DP)与最小量化误差,且保证有界失真。针对量化异构问题,引入聚类大小优化与线性融合策略以提升模型聚合精度。数值仿真验证了该方法在隐私保护与学习效用方面均优于传统拉普拉斯量化联邦学习(LaplaceSQ-FL)。
原文摘要 · Abstract (English)
Federated learning (FL) has emerged as a promising paradigm for distributed machine learning, enabling collaborative training of a global model across multiple local devices without requiring them to share raw data. Despite its advancements, FL is limited by factors such as: (i) privacy risks arising from the unprotected transmission of local model updates to the fusion center (FC) and (ii) decreased learning utility caused by heterogeneity in model quantization resolution across participating devices. Prior work typically addresses only one of these challenges because maintaining learning utility under both privacy risks and quantization heterogeneity is a non-trivial task. In this paper, our aim is therefore to improve the learning utility of a privacy-preserving FL that allows clusters of devices with different quantization resolutions to participate in each FL round. Specifically, we introduce a novel stochastic quantizer (SQ) that is designed to simultaneously achieve differential privacy (DP) and minimum quantization error. Notably, the proposed SQ guarantees bounded distortion, unlike other DP approaches. To address quantization heterogeneity, we introduce a cluster size optimization technique combined with a linear fusion approach to enhance model aggregation accuracy. Numerical simulations validate the benefits of our approach in terms of privacy protection and learning utility compared to the conventional LaplaceSQ-FL algorithm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。