arXiv:2501.19122cs.LGcs.AI2025-01NeurIPS被引 6

FedRTS用概率策略动态剪枝,提升联邦学习在弱设备上的鲁棒性。

FedRTS: Federated Robust Pruning via Combinatorial Thompson Sampling

  • 基于组合泰勒斯采样实现概率化剪枝决策,避免贪婪调整
  • 在数据异构和部分客户端参与下仍保持高性能,通信开销更低
  • 适合资源受限设备的联邦学习场景,尤其适用于不稳定的网络环境

联邦学习(FL)可在不共享数据的前提下实现跨分布式客户端的协作模型训练,但其高计算与通信开销对资源受限设备构成挑战。现有方法虽通过动态剪枝定期调整稀疏模型结构以提升效率,但仍存在贪婪调整、拓扑不稳定及通信低效等问题,导致模型鲁棒性差,在数据异构和部分客户端参与时性能下降。为此,我们提出基于组合泰勒斯采样(Combinatorial Thompson Sampling)的联邦鲁棒剪枝框架FedRTS,通过泰勒斯采样调整机制(TSAdj)利用稳定且前瞻性的概率决策,替代以往依赖不稳定、短视信息的确定性决策。大量实验表明,FedRTS在计算机视觉与自然语言处理任务中均达到当前最优性能,显著降低通信成本,尤其在数据分布异构与客户端部分可用的场景下表现优异。代码已开源:https://github.com/Little0o0/FedRTS

原文摘要 · Abstract (English)

Federated Learning (FL) enables collaborative model training across distributed clients without data sharing, but its high computational and communication demands strain resource-constrained devices. While existing methods use dynamic pruning to improve efficiency by periodically adjusting sparse model topologies while maintaining sparsity, these approaches suffer from issues such as greedy adjustments, unstable topologies, and communication inefficiency, resulting in less robust models and suboptimal performance under data heterogeneity and partial client availability. To address these challenges, we propose Federated Robust pruning via combinatorial Thompson Sampling (FedRTS), a novel framework designed to develop robust sparse models. FedRTS enhances robustness and performance through its Thompson Sampling-based Adjustment (TSAdj) mechanism, which uses probabilistic decisions informed by stable, farsighted information instead of deterministic decisions reliant on unstable and myopic information in previous methods. Extensive experiments demonstrate that FedRTS achieves state-of-the-art performance in computer vision and natural language processing tasks while reducing communication costs, particularly excelling in scenarios with heterogeneous data distributions and partial client participation. Our codes are available at: https://github.com/Little0o0/FedRTS

联邦学习模型剪枝鲁棒性采样优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。