arXiv:2502.07977cs.LGmath.OC2025-02被引 1

提出抗中间人攻击的去中心化学习算法,保障模型训练安全可靠。

RESIST: Resilient Decentralized Learning Using Consensus Gradient Descent

  • 采用多步共识梯度下降与鲁棒统计筛选,抵御通信链路篡改
  • 在强凸、PL及非凸问题中实现算法与统计收敛
  • 适合高风险通信环境下的分布式模型训练,如物联网

经验风险最小化(ERM)是现代机器学习的核心,优化理论的进步确保了高效且可证明的学习率。但在隐私、内存、计算和通信受限场景下,数据需在联网设备间分散处理。许多应用中缺乏中心服务器,要求去中心化学习算法具备高效性与鲁棒性。然而,去中心化学习面临更大攻击面,尤其易受中间人(MITM)攻击影响——攻击者可篡改传输信息,注入恶意更新,导致模型偏离预期的ERM解。本文提出RESIST(Resilient dEcentralized learning using conSensus gradIent deScenT),一种针对通信链路被恶意篡改的鲁棒优化算法。不同于现有方法(仅收敛至解附近、强凸问题无线性收敛保证、样本增大时统计不一致),RESIST通过多步共识梯度下降框架与基于鲁棒统计的筛选机制,统一解决了三类缺陷,在强凸、Polyak-Lojasiewicz(PL)及非凸问题上均实现算法与统计收敛。实验验证其在多种攻击策略、筛选方法与损失函数下的鲁棒性与可扩展性。

原文摘要 · Abstract (English)

Empirical risk minimization (ERM) is a cornerstone of modern machine learning (ML), supported by advances in optimization theory that ensure efficient solutions with provable algorithmic and statistical learning rates. Privacy, memory, computation, and communication constraints necessitate data collection, processing, and storage across network-connected devices. In many applications, networks operate in decentralized settings where a central server cannot be assumed, requiring decentralized ML algorithms that are efficient and resilient. Decentralized learning, however, faces significant challenges, including an increased attack surface. This paper focuses on the man-in-the-middle (MITM) attack, wherein adversaries exploit communication vulnerabilities to inject malicious updates during training, potentially causing models to deviate from their intended ERM solutions. To address this challenge, we propose RESIST (Resilient dEcentralized learning using conSensus gradIent deScenT), an optimization algorithm designed to be robust against adversarially compromised communication links, where transmitted information may be arbitrarily altered before being received. Unlike existing adversarially robust decentralized learning methods, which often (i) guarantee convergence only to a neighborhood of the solution, (ii) lack guarantees of linear convergence for strongly convex problems, or (iii) fail to ensure statistical consistency as sample sizes grow, RESIST overcomes all three limitations. It achieves algorithmic and statistical convergence for strongly convex, Polyak-Lojasiewicz, and nonconvex ERM problems by employing a multistep consensus gradient descent framework and robust statistics-based screening methods to mitigate the impact of MITM attacks. Experimental results demonstrate the robustness and scalability of RESIST across attack strategies, screening methods, and loss functions.

去中心化学习安全学习鲁棒优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。