提出高效个性化联邦学习剪枝框架,解决设备资源不足与模型偏差问题。
SubFLOT: Submodel Extraction for Efficient and Personalized Federated Learning via Optimal Transport

- 基于最优传输原理,用历史模型模拟数据分布实现无数据剪枝。
- 在多个数据集上相比现有方法准确率提升1.2%-3.8%,收敛更稳定。
- 适合边缘设备部署,兼顾效率与个性化,尤其适用于异构场景。
联邦学习可在保护数据隐私的前提下实现协作建模,但其实际应用受系统和统计异构性制约。现有联邦网络剪枝方法面临困境:服务端剪枝缺乏个性化,客户端剪枝对资源受限设备计算负担过重。此外,剪枝过程导致异构子模型间参数差异显著,破坏训练稳定性并阻碍全局收敛。为此,我们提出SubFLOT,一种基于最优传输的服务器端个性化联邦剪枝框架。SubFLOT引入增强型最优传输剪枝(OTP)模块,将历史客户端模型视为局部数据分布的代理,将剪枝任务建模为Wasserstein距离最小化问题,从而在不访问原始数据的情况下生成定制化子模型。同时,为缓解参数偏差,设计基于缩放的自适应正则化(SAR)模块,根据客户端剪枝率动态调整对子模型偏离全局模型的惩罚强度。大量实验表明,SubFLOT持续且显著优于当前最优方法,在多个数据集上准确率提升1.2%-3.8%,展现出在资源受限边缘设备上部署高效个性化模型的巨大潜力。
原文摘要 · Abstract (English)
Federated Learning (FL) enables collaborative model training while preserving data privacy, but its practical deployment is hampered by system and statistical heterogeneity. While federated network pruning offers a path to mitigate these issues, existing methods face a critical dilemma: server-side pruning lacks personalization, whereas client-side pruning is computationally prohibitive for resource-constrained devices. Furthermore, the pruning process itself induces significant parametric divergence among heterogeneous submodels, destabilizing training and hindering global convergence. To address these challenges, we propose SubFLOT, a novel framework for server-side personalized federated pruning. SubFLOT introduces an Optimal Transport-enhanced Pruning (OTP) module that treats historical client models as proxies for local data distributions, formulating the pruning task as a Wasserstein distance minimization problem to generate customized submodels without accessing raw data. Concurrently, to counteract parametric divergence, our Scaling-based Adaptive Regularization (SAR) module adaptively penalizes a submodel's deviation from the global model, with the penalty's strength scaled by the client's pruning rate. Comprehensive experiments demonstrate that SubFLOT consistently and substantially outperforms state-of-the-art methods, underscoring its potential for deploying efficient and personalized models on resource-constrained edge devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。