通过剪枝与客户端选择,提升资源受限下的联邦边缘学习泛化能力
Closing the Generalization Gap in Parameter-efficient Federated Edge Learning
- 基于信息论构建泛化误差分析,揭示局部泛化与全局收敛的权衡关系
- 联合优化剪枝率、客户端选择与资源分配,在能耗延迟约束下提升性能
- 适用于边缘AI场景中数据异构、设备资源有限的高效协同训练
联邦边缘学习(FEEL)为边缘人工智能提供了协作训练并保护数据隐私的可行方案。然而,受限且异构的本地数据集以及资源受限的部署环境严重削弱了模型泛化能力与资源利用率,导致学习性能下降。为此,我们提出一种参数高效的FEEL框架,联合利用模型剪枝与客户端选择以应对这些挑战。首先,我们推导出一个信息论驱动的泛化性陈述,刻画训练与测试函数损失之间的差距,并将其嵌入收敛性分析中,发现更大的局部泛化性会损害全局收敛。随后,我们构建了一个考虑泛化性的平均平方梯度范数上界最小化问题,联合优化剪枝率、客户端选择及通信-计算资源分配,在能量与延迟约束下实现最优。尽管该问题是非凸混合整数规划,但仍可通过交替优化算法高效求解。大量实验表明,所提方案在学习性能上优于现有先进基线,验证了将泛化感知分析与系统级优化结合在高效FEEL中的有效性。
原文摘要 · Abstract (English)
Federated edge learning (FEEL) provides a promising foundation for edge artificial intelligence (AI) by enabling collaborative model training while preserving data privacy. However, limited and heterogeneous local datasets, as well as resource-constrained deployment, severely degrade both model generalization and resource utilization, leading to a compromised learning performance. Therefore, we propose a parameter-efficient FEEL framework that jointly leverages model pruning and client selection to tackle such challenges. First, we derive an information-theoretic generalization statement that characterizes the discrepancy between training and testing function losses and embed it into the convergence analysis. It reveals that a larger local generalization statement can undermine the global convergence. Then, we formulate a generalization-aware average squared gradient norm bound minimization problem, by jointly optimizing the pruning ratios, client selection, and communication-computation resources under energy and delay constraints. Despite its non-convexity, the resulting mixed-integer problem is efficiently solved via an alternating optimization algorithm. Extensive experiments demonstrate that the proposed design achieves superior learning performance than state-of-the-art baselines, validating the effectiveness of coupling generalization-aware analysis with system-level optimization for efficient FEEL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。