通过边缘模型叠加，让联邦学习在设备上训练更大模型。
EMO: Edge Model Overlays to Scale Model Size in Federated Learning
- 在设备与服务器间引入边缘模型叠加层，构建更大集成模型。
- 相比传统联邦学习，准确率提升最高17.77%；相比分片联邦学习，通信开销降低7.17倍。
- 适合资源受限但需大模型的边缘场景，如智能终端协同训练。
联邦学习（FL）在分布式数据的边缘设备上训练模型，但设备的计算与内存限制制约了大模型的训练。分片联邦学习（SFL）将模型分布于设备与服务器之间以缓解此问题，却导致数据流高度耦合，引发计算瓶颈和高通信成本。本文提出EMO，通过在设备与服务器间引入边缘模型叠加（Edge Model Overlay），在不改变原有联邦学习流程的前提下实现模型规模扩展。核心创新为增强型联邦学习（AFL），通过连接原始小型FL模型与叠加层中训练的模型，实现水平或垂直扩展。其包含三个关键模块：层级激活重放缓存，解耦AFL与FL；收敛感知通信控制器，优化通信开销；以及集成推理模块。真实原型评估表明，相较于传统FL，EMO准确率最高提升17.77%；相较SFL，通信成本降低最高7.17倍，训练时间减少最多6.9倍。
原文摘要 · Abstract (English)
Federated Learning (FL) trains machine learning models on edge devices with distributed data. However, the computational and memory limitations of these devices restrict the training of large models using FL. Split Federated Learning (SFL) addresses this challenge by distributing the model across the device and server, but it introduces a tightly coupled data flow, leading to computational bottlenecks and high communication costs. We propose EMO as a solution to enable the training of large models in FL while mitigating the challenges of SFL. EMO introduces Edge Model Overlay(s) between the device and server, enabling the creation of a larger ensemble model without modifying the FL workflow. The key innovation in EMO is Augmented Federated Learning (AFL), which builds an ensemble model by connecting the original (smaller) FL model with model(s) trained in the overlay(s) to facilitate horizontal or vertical scaling. This is accomplished through three key modules: a hierarchical activation replay cache to decouple AFL from FL, a convergence-aware communication controller to optimize communication overhead, and an ensemble inference module. Evaluations on a real-world prototype show that EMO improves accuracy by up to 17.77% compared to FL, and reduces communication costs by up to 7.17x and decreases training time by up to 6.9x compared to SFL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。