通过拆分模型降低隐私噪声,提升联邦学习的隐私与精度平衡。
Federated Learning with Enhanced Privacy via Model Splitting and Random Client Participation
- 将模型分为本地私有部分和共享公共部分,仅对公共部分加噪。
- 理论证明随机参与与数据采样可显著减少所需噪声量。
- 实验证明在强隐私保护下仍能训练高精度模型,适合隐私敏感场景。
联邦学习常采用差分隐私(DP)保护客户端数据,但为保障隐私而添加的噪声会严重降低模型精度。为此,我们提出一种新型框架MS-PAFL,结合结构化模型拆分与统计隐私放大机制。每个客户端将模型分为本地保留的私有子模型和用于全局聚合的公共子模型,仅对公共子模型注入校准后的高斯噪声,从而限制噪声影响并保持本地模型性能。我们进一步提供严格的理论分析,刻画了在该架构下随机客户端参与与本地数据子采样共同带来的联合隐私放大效果。分析给出了单轮与累计隐私损失的紧致上界,证明了MS-PAFL可显著降低满足目标隐私水平所需的噪声量。大量实验验证了理论结果,表明MS-PAFL始终实现更优的隐私-效用权衡,并可在强隐私保障下训练出高精度模型。
原文摘要 · Abstract (English)
Federated Learning (FL) often adopts differential privacy (DP) to protect client data, but the added noise required for privacy guarantees can substantially degrade model accuracy. To resolve this challenge, we propose model-splitting privacy-amplified federated learning (MS-PAFL), a novel framework that combines structural model splitting with statistical privacy amplification. In this framework, each client's model is partitioned into a private submodel, retained locally, and a public submodel, shared for global aggregation. The calibrated Gaussian noise is injected only into the public submodel, thereby confining its adverse impact while preserving the utility of the local model. We further present a rigorous theoretical analysis that characterizes the joint privacy amplification achieved through random client participation and local data subsampling under this architecture. The analysis provides tight bounds on both single-round and total privacy loss, demonstrating that MS-PAFL significantly reduces the noise necessary to satisfy a target privacy protection level. Extensive experiments validate our theoretical findings, showing that MS-PAFL consistently attains a superior privacy-utility trade-off and enables the training of highly accurate models under strong privacy guarantees.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。