arXiv:2501.00693cs.DCcs.LG2025-01被引 5

解决跨端边云联邦学习中的模型异构与知识传递难题,实现更高效、鲁棒的协同训练。

Beyond Model Scale Limits: End-Edge-Cloud Federated Learning with Self-Rectified Knowledge Agglomeration

  • 通过桥接样本实现相邻节点间动态知识迁移,突破设备算力限制。
  • 引入自修正机制,防止云端模型因异质数据出现优化偏差。
  • 适配移动节点和网络波动,适合资源分布不均的真实边缘场景。

端-边-云协同(EECC)为人工智能模型训练提供了新范式,可在终端设备、边缘服务器与云数据中心间实现更可靠、低延迟的协作。分层联邦学习(HFL)可借助此架构,在多层级计算节点间进行模型聚合。然而,其潜力受限于EECC环境固有的异构性与动态性:所有节点受最弱终端设备的模型结构约束,形成性能瓶颈;各层级间数据分布与资源能力差异导致知识传递失衡,引发更新偏差与性能下降;同时,节点移动性与网络波动加剧了动态迁移复杂度,影响训练稳定性。为此,我们提出端-边-云联邦学习自修正知识聚合框架(FedEEC),支持从终端到云端模型逐步增大规模并增强泛化能力。核心创新包括:(1) 桥接样本在线蒸馏协议(BSBODP),通过生成桥接样本实现邻近节点间知识转移;(2) 自知识修正(SKR),对传输知识进行动态校正,避免云端模型陷入次优优化。该框架有效应对跨层级资源异构与知识传递问题,满足EECC环境下的迁移韧性需求。

原文摘要 · Abstract (English)

The rise of End-Edge-Cloud Collaboration (EECC) offers a promising paradigm for Artificial Intelligence (AI) model training across end devices, edge servers, and cloud data centers, providing enhanced reliability and reduced latency. Hierarchical Federated Learning (HFL) can benefit from this paradigm by enabling multi-tier model aggregation across distributed computing nodes. However, the potential of HFL is significantly constrained by the inherent heterogeneity and dynamic characteristics of EECC environments. Specifically, the uniform model structure bounded by the least powerful end device across all computing nodes imposes a performance bottleneck. Meanwhile, coupled heterogeneity in data distributions and resource capabilities across tiers disrupts hierarchical knowledge transfer, leading to biased updates and degraded performance. Furthermore, the mobility and fluctuating connectivity of computing nodes in EECC environments introduce complexities in dynamic node migration, further compromising the robustness of the training process. To address multiple challenges within a unified framework, we propose End-Edge-Cloud Federated Learning with Self-Rectified Knowledge Agglomeration (FedEEC), which is a novel EECC-empowered FL framework that allows the trained models from end, edge, to cloud to grow larger in size and stronger in generalization ability. FedEEC introduces two key innovations: (1) Bridge Sample Based Online Distillation Protocol (BSBODP), which enables knowledge transfer between neighboring nodes through generated bridge samples, and (2) Self-Knowledge Rectification (SKR), which refines the transferred knowledge to prevent suboptimal cloud model optimization. The proposed framework effectively handles both cross-tier resource heterogeneity and effective knowledge transfer between neighboring nodes, while satisfying the migration-resilient requirements of EECC.

联邦学习端边云协同知识蒸馏异构系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。