解决边缘计算中数据异构导致的模型漂移问题,提升低通信开销下的训练稳定性。
Mitigating Heterogeneity-Induced Drift in Hierarchical Sign-Based Federated Learning
- 设备上传梯度符号,边缘端用多数投票聚合,云端定期整合模型
- 在严重数据异构下,新算法使准确率接近全精度方法,通信量大幅降低
- 通过云端辅助梯度修正,有效抑制非消失的异构偏置,适合资源受限场景
层级联邦学习(HFL)适用于大规模无线与物联网系统,设备先与邻近边缘服务器通信再传至云端。上行带宽和延迟限制要求强烈压缩通信,一比特符号随机梯度下降在平坦联邦设置中表现良好,但在层级边缘-云架构中,尤其在集群间数据异构下,其行为尚不明确。本文提出一种符号化HFL框架:设备向边缘服务器发送二值梯度符号,边缘服务器执行多数投票,云端周期性聚合边缘模型。分析表明,集群间异构导致收敛界中存在持续偏置项,反映边缘模型向本地目标漂移。该偏置无法仅通过增加训练轮数或调整常规超参数消除。为此,我们提出\(\mathtt{DC\text{-}HierSignSGD}\),设备在取符号前进行云端辅助梯度修正。证明该预符号修正可缓解非消失的异构偏置,同时保持设备-边缘间二值通信。实验显示,在严重集群间异构下,\(\mathtt{DC\text{-}HierSignSGD}\)显著提升符号化HFL的稳定性和准确率,性能接近全精度层级SGD,但设备-边缘通信量大幅减少。
原文摘要 · Abstract (English)
Hierarchical federated learning (HFL) is well suited for large-scale wireless and Internet of Things systems, where devices communicate with nearby edge servers before reaching the cloud. In these environments, uplink bandwidth and latency impose strict communication constraints, making aggressive gradient compression essential. One-bit sign-based stochastic gradient descent methods provide an attractive solution in flat federated settings, but their behavior in hierarchical edge--cloud architectures remains insufficiently understood, especially under inter-cluster data heterogeneity. To address this gap, we develop a sign-based HFL framework in which devices transmit binary stochastic-gradient signs to edge servers, edge servers apply majority voting, and the cloud periodically aggregates edge models. Our analysis reveals that inter-cluster heterogeneity induces a persistent bias term in the convergence bound, reflecting the drift of edge models toward local objectives. This term cannot be removed by increasing the number of training rounds or by tuning standard hyperparameters alone. We therefore propose \(\mathtt{DC\text{-}HierSignSGD}\), a drift-corrected sign-based HFL algorithm in which devices apply a cloud-assisted gradient correction before taking the sign. We show that this pre-sign correction mitigates the non-vanishing heterogeneity-induced bias while preserving binary device--edge communication during the repeated local sign-update steps. Experiments under severe inter-cluster heterogeneity demonstrate that \(\mathtt{DC\text{-}HierSignSGD}\) improves the stability and accuracy of sign-based HFL and achieves performance comparable to full-precision hierarchical SGD with substantially lower device--edge communication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。