解决异构数据下联邦学习的梯度偏差问题，提升模型收敛速度与准确率。
SMoFi: Step-wise Momentum Fusion for Split Federated Learning on Heterogeneous Data
- 分步动量融合机制，同步服务器端优化器的动量缓冲区。
- 实测精度最高提升7.1%，收敛速度最快提高10.25倍。
- 适用于多客户端、深层模型和资源受限场景。
Split Federated Learning 利用中心服务器丰富的计算资源训练模型分块，但各数据孤岛间的数据异构性严重影响全局模型的收敛速度与精度。本文提出分步动量融合（SMoFi）框架，通过同步服务器端优化器的动量缓冲区，缓解由数据异构引发的梯度分歧。设计了一种时效感知对齐机制，在每一步优化中对服务器子模型的梯度更新施加约束。在多个真实数据集上的大量验证表明，SMoFi 持续提升全局模型精度（最高达7.1%），并加快收敛速度（最高达10.25倍）。且随着客户端数量增多和模型深度增加，效果更显著，特别适合资源受限环境下的模型训练。
原文摘要 · Abstract (English)
Split Federated Learning is a system-efficient federated learning paradigm that leverages the rich computing resources at a central server to train model partitions. Data heterogeneity across silos, however, presents a major challenge undermining the convergence speed and accuracy of the global model. This paper introduces Step-wise Momentum Fusion (SMoFi), an effective and lightweight framework that counteracts gradient divergence arising from data heterogeneity by synchronizing the momentum buffers across server-side optimizers. To control gradient divergence over the training process, we design a staleness-aware alignment mechanism that imposes constraints on gradient updates of the server-side submodel at each optimization step. Extensive validations on multiple real-world datasets show that SMoFi consistently improves global model accuracy (up to 7.1%) and convergence speed (up to 10.25$\times$). Furthermore, SMoFi has a greater impact with more clients involved and deeper learning models, making it particularly suitable for model training in resource-constrained contexts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。