压缩版梯度下降更难被攻击，隐私更安全。
Trustworthiness of Stochastic Gradient Descent in Distributed Learning
- 对比压缩与未压缩梯度下降的抗攻击能力。
- 压缩梯度对梯度反演攻击的防御能力显著更强。
- 发现成员推断攻击不适合作为隐私风险指标。
分布式学习（DL）通过多节点并行加速训练，提升大规模模型优化效率。随机梯度下降（SGD）是其中的核心优化算法。然而，通信瓶颈常限制其可扩展性与效率，促使压缩SGD技术广泛应用以缓解通信开销。尽管有效降低通信量，压缩SGD引入了信任问题：节点间梯度交换易受梯度反演（GradInv）和成员推断攻击（MIA）等威胁。压缩SGD的可信性尚未得到系统评估，其可靠性仍存疑问。本文首次对压缩与未压缩SGD进行可信性对比研究。通过实证分析梯度反演攻击，发现压缩SGD在防止隐私泄露方面表现显著优于未压缩版本。此外，研究结果表明，成员推断攻击可能无法准确反映分布式学习中的真实隐私风险。
原文摘要 · Abstract (English)
Distributed learning (DL) uses multiple nodes to accelerate training, enabling efficient optimization of large-scale models. Stochastic Gradient Descent (SGD), a key optimization algorithm, plays a central role in this process. However, communication bottlenecks often limit scalability and efficiency, leading to increasing adoption of compressed SGD techniques to alleviate these challenges. Despite addressing communication overheads, compressed SGD introduces trustworthiness concerns, as gradient exchanges among nodes are vulnerable to attacks like gradient inversion (GradInv) and membership inference attacks (MIA). The trustworthiness of compressed SGD remains unexplored, leaving important questions about its reliability unanswered. In this paper, we provide a trustworthiness evaluation of compressed versus uncompressed SGD. Specifically, we conducted empirical studies using GradInv attacks, revealing that compressed SGD demonstrates significantly higher resistance to privacy leakage compared to uncompressed SGD. In addition, our findings suggest that MIA may not be a reliable metric for assessing privacy risks in distributed learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。