针对异构设备的联邦学习,实现个性化压缩并保持模型兼容与稳定收敛。
CA-HFP: Curvature-Aware Heterogeneous Federated Pruning with Model Reconstruction
- 基于曲率重要性评分,为每台设备定制结构化剪枝。
- 剪枝后通过轻量重建映射回统一参数空间,精度损失小。
- 适合资源受限的边缘设备,显著降低计算与通信开销。
在异构边缘设备上的联邦学习需要个性化压缩,同时保持聚合兼容性和稳定收敛。本文提出曲率感知异构联邦剪枝(CA-HFP),使各客户端基于曲率指导的重要性评分执行设备专属的结构化剪枝,并通过轻量级重建将压缩后的子模型映射回统一的全局参数空间。我们推导了含多步本地SGD、数据异构性及剪枝扰动的联邦优化收敛界,据此设计出基于损失的剪枝准则。在FMNIST、CIFAR-10和CIFAR-100上使用VGG与ResNet架构,在不同数据异构程度下进行的大量实验表明,CA-HFP在显著降低单客户端计算与通信成本的同时,保持了模型精度,优于标准联邦训练及现有剪枝基线。
原文摘要 · Abstract (English)
Federated learning on heterogeneous edge devices requires personalized compression while preserving aggregation compatibility and stable convergence. We present Curvature-Aware Heterogeneous Federated Pruning (CA-HFP), a practical framework that enables each client perform structured, device-specific pruning guided by a curvature-informed significance score, and subsequently maps its compact submodel back into a common global parameter space via a lightweight reconstruction. We derive a convergence bound for federated optimization with multiple local SGD steps that explicitly accounts for local computation, data heterogeneity, and pruning-induced perturbations; from which a principled loss-based pruning criterion is derived. Extensive experiments on FMNIST, CIFAR-10, and CIFAR-100 using VGG and ResNet architectures under varying degrees of data heterogeneity demonstrate that CA-HFP preserves model accuracy while significantly reducing per-client computation and communication costs, outperforming standard federated training and existing pruning-based baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。