提出Delta Sum学习方法，提升去中心化学习的收敛速度与精度。
Delta Sum Learning: an approach for fast and global convergence in Gossip Learning
- 用增量求和替代传统平均，优化节点间模型聚合方式。
- 50节点时全局准确率下降比传统方法低58%。
- 适合边缘计算中需动态部署的多任务应用。
联邦学习因其安全性和计算优势而广受欢迎。随着网络边缘设备算力增强，去中心化的Gossip学习进一步移除了中心化聚合，依赖节点间直接通信。然而，联邦与Gossip学习普遍采用的平均聚合方式在模型精度和全局收敛性上表现不佳。此外，现有方案难以通过声明式方式（如Kubernetes清单）在边缘部署学习任务。本文提出Delta Sum Learning，改进Gossip学习中的基础聚合操作，并构建基于开放应用模型的去中心化编排框架，支持动态节点发现与意图驱动的多工作负载部署。实验表明，在10节点拓扑下性能与替代方法相当，但在扩展至50节点时，全局准确率下降降低58%。整体表现出强全局收敛性，准确率损失随拓扑规模呈对数增长，优于替代方法在有限连通性下的线性损失。
原文摘要 · Abstract (English)
Federated Learning is a popular approach for distributed learning due to its security and computational benefits. With the advent of powerful devices in the network edge, Gossip Learning further decentralizes Federated Learning by removing centralized integration and relying fully on peer to peer updates. However, the averaging methods generally used in both Federated and Gossip Learning are not ideal for model accuracy and global convergence. Additionally, there are few options to deploy Learning workloads in the edge as part of a larger application using a declarative approach such as Kubernetes manifests. This paper proposes Delta Sum Learning as a method to improve the basic aggregation operation in Gossip Learning, and implements it in a decentralized orchestration framework based on Open Application Model, which allows for dynamic node discovery and intent-driven deployment of multi-workload applications. Evaluation results show that Delta Sum performance is on par with alternative integration methods for 10 node topologies, but results in a 58% lower global accuracy drop when scaling to 50 nodes. Overall, it shows strong global convergence and a logarithmic loss of accuracy with increasing topology size compared to a linear loss for alternatives under limited connectivity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。