提出自适应聚合方法,提升去中心化学习抗攻击能力。
Resilient Peer-to-peer Learning based on Adaptive Aggregation
- 根据邻居模型和私有数据动态计算聚合权重
- 在非凸损失与非独立同分布数据下实现参数收敛
- 适用于抵御恶意节点的分布式学习场景
去中心化网络中的协同学习虽能避免中心服务器单点故障风险,但面对恶意参与者注入误导信息的威胁,其鲁棒性面临挑战。尤其在非凸损失函数与非独立同分布(non-iid)数据条件下,这一问题更为严峻。本文提出一种针对此类场景的鲁棒聚合机制,旨在促进各参与方学习过程的一致性。聚合权重通过优化过程确定,基于邻居模型及本地私有数据计算的损失函数,兼顾了隐私保护。理论分析证明,在非凸损失与non-iid数据下参数仍可收敛。三项不同机器学习任务的实证评估验证了该方法的有效性,涵盖多种攻击模型,相比现有方法显著提升准确率。
原文摘要 · Abstract (English)
Collaborative learning in peer-to-peer networks offers the benefits of distributed learning while mitigating the risks associated with single points of failure inherent in centralized servers. However, adversarial workers pose potential threats by attempting to inject malicious information into the network. Thus, ensuring the resilience of peer-to-peer learning emerges as a pivotal research objective. The challenge is exacerbated in the presence of non-convex loss functions and non-iid data distributions. This paper introduces a resilient aggregation technique tailored for such scenarios, aimed at fostering similarity among peers' learning processes. The aggregation weights are determined through an optimization procedure, and use the loss function computed using the neighbor's models and individual private data, thereby addressing concerns regarding data privacy in distributed machine learning. Theoretical analysis demonstrates convergence of parameters with non-convex loss functions and non-iid data distributions. Empirical evaluations across three distinct machine learning tasks support the claims. The empirical findings, which encompass a range of diverse attack models, also demonstrate improved accuracy when compared to existing methodologies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。