arXiv:2412.07216cs.DCcs.LG2024-12被引 4

提出可学习的稀疏化方法,解决边缘计算中设备差异与数据不均衡问题。

Learnable Sparse Customization in Heterogeneous Edge Computing

  • 基于重要性自适应生成个性化稀疏模式,减少人工规则依赖。
  • 在非独立同分布数据下提升精度1.28%至59.34%,训练时间减少超68.80%。
  • 适合资源异构、数据分布不均的边缘联邦学习场景使用。

为有效管理网络边缘海量分布式数据,联邦学习(FL)已成为跨数据孤岛的边缘计算范式。然而,FL仍面临两大挑战:系统异构性(边缘设备硬件资源差异)和统计异构性(非独立同分布数据)。尽管稀疏化可为不同客户端提取多样化子模型,但多数稀疏联邦学习方法仅采用人为设定的固定规则或启发式策略剪枝部分参数,导致稀疏化僵化且性能不佳。本文提出可学习的个性化稀疏联邦学习(FedLPS),通过重要性关联模式与自适应稀疏比例,实现对异构稀疏模型的可学习定制,同时应对系统与统计异构性。具体而言,FedLPS学习模型单元在本地数据表征中的重要性,进而以最少启发式生成基于重要性的稀疏模式,精准提取非独立同分布环境下的个性化特征。此外,设计提示置信度上限方差(P-UCBV)算法,自适应确定稀疏比例,学习不同设备能力与非独立同分布数据的叠加效应,实现资源自适应与高精度平衡。大量实验表明,相比现有方法,FedLPS在精度与训练成本上均有显著提升,精度提高1.28%–59.34%,运行时间减少超过68.80%。

原文摘要 · Abstract (English)

To effectively manage and utilize massive distributed data at the network edge, Federated Learning (FL) has emerged as a promising edge computing paradigm across data silos. However, FL still faces two challenges: system heterogeneity (i.e., the diversity of hardware resources across edge devices) and statistical heterogeneity (i.e., non-IID data). Although sparsification can extract diverse submodels for diverse clients, most sparse FL works either simply assign submodels with artificially-given rigid rules or prune partial parameters using heuristic strategies, resulting in inflexible sparsification and poor performance. In this work, we propose Learnable Personalized Sparsification for heterogeneous Federated learning (FedLPS), which achieves the learnable customization of heterogeneous sparse models with importance-associated patterns and adaptive ratios to simultaneously tackle system and statistical heterogeneity. Specifically, FedLPS learns the importance of model units on local data representation and further derives an importance-based sparse pattern with minimal heuristics to accurately extract personalized data features in non-IID settings. Furthermore, Prompt Upper Confidence Bound Variance (P-UCBV) is designed to adaptively determine sparse ratios by learning the superimposed effect of diverse device capabilities and non-IID data, aiming at resource self-adaptation with promising accuracy. Extensive experiments show that FedLPS outperforms status quo approaches in accuracy and training costs, which improves accuracy by 1.28%-59.34% while reducing running time by more than 68.80%.

联邦学习边缘计算稀疏化异构

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。