改进去中心化学习中异常数据知识的传播效率
Topology-Aware Knowledge Propagation in Decentralized Learning
- 根据网络拓扑设计感知聚合策略，提升知识传递效果
- 在拓扑中引入异常数据后，准确率平均提升123%
- 适合研究分布式学习与异构数据传播的开发者
去中心化学习允许在自然分布的数据间协作训练模型，无需集中协调或维护全局模型。设备以任意通信拓扑连接，仅能与邻近设备通信。每个设备通过本地数据训练并结合邻居模型进行聚合，实现知识在拓扑中的逐轮传播。本文重点关注分布外（OOD）知识的传播。发现主流去中心化学习算法难以有效将OOD知识传遍所有设备，且知识传播效果受OOD数据位置和拓扑结构显著影响。为此，提出拓扑感知聚合策略，显著加速OOD知识传播。相比无拓扑感知基线，在拓扑内各模型上平均提升123%的OOD数据准确率。
原文摘要 · Abstract (English)
Decentralized learning enables collaborative training of models across naturally distributed data without centralized coordination or maintenance of a global model. Instead, devices are organized in arbitrary communication topologies, in which they can only communicate with neighboring devices. Each device maintains its own local model by training on its local data and integrating new knowledge via model aggregation with neighbors. Therefore, knowledge is propagated across the topology via successive aggregation rounds. We study, in particular, the propagation of out-of-distribution (OOD) knowledge. We find that popular decentralized learning algorithms struggle to propagate OOD knowledge effectively to all devices. Further, we find that both the location of OOD data within a topology, and the topology itself, significantly impact OOD knowledge propagation. We then propose topology-aware aggregation strategies to accelerate (OOD) knowledge propagation across devices. These strategies improve OOD data accuracy, compared to topology-unaware baselines, by 123% on average across models in a topology.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。