SFL中数据异构导致模型遗忘早期标签，新方法有效缓解此问题。
Data Heterogeneity and Forgotten Labels in Split Federated Learning
- 通过分层训练机制，客户端与服务器协同优化模型
- 模型对序列末尾标签表现更好，早期标签易被遗忘
- 提出Hydra方法，借鉴多头网络思想提升记忆稳定性
在分割联邦学习（SFL）中，客户端与服务器协作训练模型，将模型分为两部分：第一部分在客户端本地训练并定期聚合，第二部分由服务器根据客户端传来的中间激活值顺序处理。本文研究在数据异构条件下SFL中的灾难性遗忘（CF）现象。由于SFL结构特性，第一部分的本地更新可能偏离全局最优，而第二部分对处理顺序敏感，类似持续学习中的遗忘问题。实验发现，模型在序列末尾出现的类别上表现更优。本文重点分析了服务器处理顺序和切分层的影响，并提出Hydra方法，受多头神经网络启发，专为SFL设计。大量数值实验表明，Hydra优于现有基线及文献方法。
原文摘要 · Abstract (English)
In Split Federated Learning (SFL), the clients collaboratively train a model with the help of a server by splitting the model into two parts. Part-1 is trained locally at each client and aggregated by the aggregator at the end of each round. Part-2 is trained at a server that sequentially processes the intermediate activations received from each client. We study the phenomenon of catastrophic forgetting (CF) in SFL in the presence of data heterogeneity. In detail, due to the nature of SFL, local updates of part-1 may drift away from global optima, while part-2 is sensitive to the processing sequence, similar to forgetting in continual learning (CL). Specifically, we observe that the trained model performs better in classes (labels) seen at the end of the sequence. We investigate this phenomenon with emphasis on key aspects of SFL, such as the processing order at the server and the cut layer. Based on our findings, we propose Hydra, a novel mitigation method inspired by multi-head neural networks and adapted for the SFL setting. Extensive numerical evaluations show that Hydra outperforms baselines and methods from the literature.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。