通过相关噪声设计，让分布式学习在保护隐私的同时提升模型性能。
Optimizing Privacy-Utility Trade-off in Decentralized Learning with Generalized Correlated Noise
- 用网络拓扑和权重生成相关噪声，实现全局噪声抵消
- 实验显示噪声抵消率更高，模型准确率显著提升
- 适合关注隐私保护与模型效果平衡的研究者
去中心化学习允许各参与方在无中心服务器的情况下通过本地计算和点对点通信协同训练共享模型。尽管数据保留在本地，但交换本地模型仍可能泄露训练数据的隐私信息。为防范隐私攻击，通常在每个节点向邻居发送模型前注入人工随机噪声。然而，累积噪声会损害模型性能。本文提出CorN-DSGD，一种基于协方差的跨节点相关噪声生成框架，可统一多种前沿方法。利用网络拓扑与混合权重，优化噪声协方差以实现全网噪声抵消。实验表明，该方法比现有成对相关方案更有效降低噪声，同时在严格的隐私保障下提升模型表现。
原文摘要 · Abstract (English)
Decentralized learning enables distributed agents to collaboratively train a shared machine learning model without a central server, through local computation and peer-to-peer communication. Although each agent retains its dataset locally, sharing local models can still expose private information about the local training datasets to adversaries. To mitigate privacy attacks, a common strategy is to inject random artificial noise at each agent before exchanging local models between neighbors. However, this often leads to utility degradation due to the negative effects of cumulated artificial noise on the learning algorithm. In this work, we introduce CorN-DSGD, a novel covariance-based framework for generating correlated privacy noise across agents, which unifies several state-of-the-art methods as special cases. By leveraging network topology and mixing weights, CorN-DSGD optimizes the noise covariance to achieve network-wide noise cancellation. Experimental results show that CorN-DSGD cancels more noise than existing pairwise correlation schemes, improving model performance under formal privacy guarantees.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。