用矩阵分解统一分析去中心化学习的隐私,提升保护效果。
Unified Privacy Guarantees for Decentralized Learning via Matrix Factorization
- 将矩阵分解用于分析去中心化学习中的噪声相关性
- 在真实和合成图上隐私-效用权衡优于现有方法
- 适合关注隐私保护与模型性能平衡的研究者
去中心化学习(DL)通过网络中邻居间迭代平均本地更新,实现无需共享原始数据的协同建模,具备良好可扩展性且能保持数据本地化。强隐私保障通常依赖差分隐私(DP),已有研究发现通过点对点通信传播噪声可增强隐私。然而实践中隐私-效用权衡常劣于集中式训练,可能源于当前针对去中心化学习的差分隐私计数方法局限。本文表明,基于矩阵分解(MF)的集中式隐私计数新进展可拓展至去中心化场景,通过推广现有MF结果,将标准DL算法与常见信任模型统一建模。该方法为现有DP-DL算法提供更紧的隐私计数,并为设计新算法提供理论基础。为验证方法有效性,我们提出MAFALDA-SGD——一种基于传闻机制的去中心化学习算法,引入用户级相关噪声,在合成及真实世界图上表现优于现有方法。
原文摘要 · Abstract (English)
Decentralized Learning (DL) enables users to collaboratively train models without sharing raw data by iteratively averaging local updates with neighbors in a network graph. This setting is increasingly popular for its scalability and its ability to keep data local under user control. Strong privacy guarantees in DL are typically achieved through Differential Privacy (DP), with results showing that DL can even amplify privacy by disseminating noise across peer-to-peer communications. Yet in practice, the observed privacy-utility trade-off often appears worse than in centralized training, which may be due to limitations in current DP accounting methods for DL. In this paper, we show that recent advances in centralized DP accounting based on Matrix Factorization (MF) for analyzing temporal noise correlations can also be leveraged in DL. By generalizing existing MF results, we show how to cast both standard DL algorithms and common trust models into a unified formulation. This yields tighter privacy accounting for existing DP-DL algorithms and provides a principled way to develop new ones. To demonstrate the approach, we introduce MAFALDA-SGD, a gossip-based DL algorithm with user-level correlated noise that outperforms existing methods on synthetic and real-world graphs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。