提出两种谱分片采样策略，提升异构设备联邦学习效率。
On Sampling Strategies for Spectral Model Sharding
- 基于优化问题设计两种采样策略，保证权重无偏或最小逼近误差。
- 在多个数据集上验证，性能优于传统方法。
- 适合资源受限的边缘设备联邦学习场景。
联邦学习中的异构客户端问题近年来受到广泛关注。谱模型分片（spectral model sharding）——即基于奇异值分解将模型参数划分为低秩矩阵——是提升此类场景下设备端训练效率的一种方案。本文提出两种针对此类分片的采样策略，分别源于特定优化问题的解：第一种生成原始权重的无偏估计，第二种旨在最小化平方逼近误差。我们讨论了这两种估计器如何融入联邦学习流程，并分析了本地训练中出现的实际问题。实验表明，两种方法在多个常用数据集上均能提升性能。
原文摘要 · Abstract (English)
The problem of heterogeneous clients in federated learning has recently drawn a lot of attention. Spectral model sharding, i.e., partitioning the model parameters into low-rank matrices based on the singular value decomposition, has been one of the proposed solutions for more efficient on-device training in such settings. In this work, we present two sampling strategies for such sharding, obtained as solutions to specific optimization problems. The first produces unbiased estimators of the original weights, while the second aims to minimize the squared approximation error. We discuss how both of these estimators can be incorporated in the federated learning loop and practical considerations that arise during local training. Empirically, we demonstrate that both of these methods can lead to improved performance on various commonly used datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。