提出一种无需损失精度的分布式学习框架，实现高效隐私保护训练。
Traversal Learning: A Lossless And Efficient Distributed Learning Framework
- 模型在前向传播中遍历各节点，反向传播由中心节点统一执行。
- 在六大数据集上表现超越传统方法，最高提升达7.85%准确率。
- 适合对数据隐私和性能要求高的实际场景，如医疗与金融领域。
本文提出了一种新型分布式学习框架——遍历学习（Traversal Learning, TL），旨在解决联邦学习（FL）、分割学习（SL）及分段联邦学习（SFL）中存在的性能下降问题。传统FL因聚合时采用平均策略导致精度下降，而SL与SFL则因各分割网络独立更新梯度引发损失上升。TL通过在前向传播中让模型遍历各节点，并在协调器处执行反向传播，实现了分布式环境下的集中式学习效果。协调器负责生成虚拟批次并规划模型在前向传播中的节点访问顺序，使其与批次内数据的有序索引对齐。我们在六个涵盖不同领域特征的数据集上进行了实验，结果表明：TL在推理精度上与经典集中式学习相当，且显著优于其他分布式方法。在独立同分布（IID）数据集上，准确率提升7.85%；非独立同分布（non-IID）数据集上，宏平均F1分数提升1.06%；文本分类任务中准确率提升2.60%；医学与金融数据集的AUC分别提升3.88%和4.54%。该方法在有效保护数据隐私的同时维持高性能，是分布式学习的重要进展。代码已开源：https://github.com/neouly-inc/Traversal-Learning。
原文摘要 · Abstract (English)
In this paper, we introduce Traversal Learning (TL), a novel approach designed to address the problem of decreased quality encountered in popular distributed learning (DL) paradigms such as Federated Learning (FL), Split Learning (SL), and SplitFed Learning (SFL). Traditional FL experiences from an accuracy drop during aggregation due to its averaging function, while SL and SFL face increased loss due to the independent gradient updates on each split network. TL adopts a unique strategy where the model traverses the nodes during forward propagation (FP) and performs backward propagation (BP) on the orchestrator, effectively implementing centralized learning (CL) principles within a distributed environment. The orchestrator is tasked with generating virtual batches and planning the sequential node visits of the model during FP, aligning them with the ordered index of the data within these batches. We conducted experiments on six datasets representing diverse characteristics across various domains. Our evaluation demonstrates that TL is on par with classic CL approaches in terms of accurate inference, thereby offering a viable and robust solution for DL tasks. TL outperformed other DL methods and improved accuracy by 7.85% for independent and identically distributed (IID) datasets, macro F1-score by 1.06% for non-IID datasets, accuracy by 2.60% for text classification, and AUC by 3.88% and 4.54% for medical and financial datasets, respectively. By effectively preserving data privacy while maintaining performance, TL represents a significant advancement in DL methodologies. The implementation of TL is available at https://github.com/neouly-inc/Traversal-Learning
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。