针对联邦学习中数据与设备差异,提出动态分组剪枝框架,提升效率与公平性。
CA-AFP: Cluster-Aware Adaptive Federated Pruning
- 按客户端分组后,为每组定制剪枝策略,结合权重大小、组内一致性等判断剪枝参数。
- 在两个活动识别数据集上,精度更高且客户端间性能差距更小,通信量比密集聚类方法降低约40%。
- 适合资源受限设备上的高效公平联邦学习,尤其适用于非独立同分布数据场景。
联邦学习在真实部署中面临客户端间统计异质性与资源受限设备带来的系统异质性挑战。现有聚类方法缓解统计异质性,剪枝技术提升内存与通信效率,但二者常被孤立研究。本文提出统一框架CA-AFP,通过聚类特定的模型剪枝联合应对两类问题。首先将客户端聚类,再在训练中自适应地为每个簇剪枝。核心创新包括:(1) 聚类感知的重要性评分机制,融合权重大小、组内一致性与梯度一致性以识别可剪参数;(2) 迭代式剪枝策略,逐步移除参数并支持通过权重再生实现模型自愈。在两个主流人体活动识别基准——UCI HAR和WISDM——上,采用自然用户划分的联邦设置进行评估。结果表明,CA-AFP在预测精度、客户端间公平性与通信效率之间取得良好平衡。相比剪枝基线,其在有限微调下持续提升精度并降低客户端性能差异;相比密集聚类方法,通信开销显著减少约40%。对不同非独立同分布(Non-IID)程度的数据也表现出强鲁棒性。消融实验分析了聚类、剪枝调度与评分机制的影响,为设计高效自适应联邦学习系统提供实践启示。
原文摘要 · Abstract (English)
Federated Learning (FL) faces major challenges in real-world deployments due to statistical heterogeneity across clients and system heterogeneity arising from resource-constrained devices. While clustering-based approaches mitigate statistical heterogeneity and pruning techniques improve memory and communication efficiency, these strategies are typically studied in isolation. We propose CA-AFP, a unified framework that jointly addresses both challenges by performing cluster-specific model pruning. In CA-AFP, clients are first grouped into clusters, and a separate model for each cluster is adaptively pruned during training. The framework introduces two key innovations: (1) a cluster-aware importance scoring mechanism that combines weight magnitude, intra-cluster coherence, and gradient consistency to identify parameters for pruning, and (2) an iterative pruning schedule that progressively removes parameters while enabling model self-healing through weight regrowth. We evaluate CA-AFP on two widely used human activity recognition benchmarks, UCI HAR and WISDM, under natural user-based federated partitions. Experimental results demonstrate that CA-AFP achieves a favorable balance between predictive accuracy, inter-client fairness, and communication efficiency. Compared to pruning-based baselines, CA-AFP consistently improves accuracy and lower performance disparity across clients with limited fine-tuning, while requiring substantially less communication than dense clustering-based methods. It also shows robustness to different Non-IID levels of data. Finally, ablation studies analyze the impact of clustering, pruning schedules and scoring mechanism offering practical insights into the design of efficient and adaptive FL systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。