针对边缘系统延迟约束，协同训练多模型以提升精度。
Collate: Collaborative Neural Network Learning for Latency-Critical Edge Systems
- 动态调整本地模型结构，在延迟限制下保持高精度。
- 单次训练实现多系统适配，平均精度提升1.96%~3.09%。
- 适合对延迟敏感的异构边缘设备部署场景。
联邦学习（FL）使多个客户端协作训练模型，在保护数据隐私的同时扩大各客户端的训练数据以提高精度。然而，在实时边缘系统中，设备异构性严重制约了推理模型性能。现有优化聚焦训练效率，却未解决推理速度问题，尤其在延迟约束下。本文提出Collate框架，协同学习满足多个边缘系统延迟约束的异构模型。设计动态零化-恢复方法，调整本地模型结构以适应延迟限制；引入原型修正联邦聚合方案，统一训练过程下聚合不同延迟要求的模型，维持高精度。大量实验表明，在延迟约束下，扩展模型平均精度提升1.96%，压缩模型平均提升3.09%，几乎无额外训练开销。相关代码与数据将公开于https://github.com/ntuliuteam/Collate。
原文摘要 · Abstract (English)
Federated Learning (FL) empowers multiple clients to collaboratively learn a model, enlarging the training data of each client for high accuracy while protecting data privacy. However, when deploying FL in real-time edge systems, the heterogeneity of devices among systems has a severe impact on the performance of the inferred model. Existing optimizations on FL focus on improving the training efficiency but fail to speed up inference, especially when there is a latency constraint. In this work, we propose Collate, a novel training framework that collaboratively learns heterogeneous models to meet the latency constraints of multiple edge systems simultaneously. We design a dynamic zeroizing-recovering method to adjust each local model architecture for high accuracy under its latency constraint. A proto-corrected federated aggregation scheme is also introduced to aggregate all heterogeneous local models, satisfying the latency constraint of different systems with only one training process and maintaining high accuracy. Extensive experiments indicate that, compared to state-of-the-art methods and under a latency constraint, our extended models can improve the accuracy by 1.96% on average, and our shrunk models can also obtain a 3.09% accuracy improvement on average, with almost no extra training overhead. The related codes and data will be available at https://github.com/ntuliuteam/Collate
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。