用贝叶斯优化选择客户端代表性数据子集,提升个性化联邦学习效率
Bayesian Coreset Optimization for Personalized Federated Learning
- 基于贝叶斯方法构建客户端数据压缩子集,仅传输关键样本
- 理论证明误差上界为O(n_k^(-2β/(2β+Λ)) log²δ'(n_k)),下界为O(n_k^(-2β/(2β+Λ)))
- 在医疗等数据稀缺场景中优于随机采样与子模优化方法
在联邦学习中,多个客户端各自更新模型并上传至中央服务器,但每次训练使用全部本地数据效率低下。为此,我们提出\methodprop:一种基于加权核心集的个性化联邦学习框架,仅通过各客户端的核心集代表性数据点进行更新,而非全量数据。理论分析表明,平均泛化误差在对数范围内达到极小极大最优(上界为\mathcal{O}(n_k^{-\frac{2 β}{2 β+\boldsymbolΛ}} \log ^{2 δ^{\prime}}(n_k))),下界为\mathcal{O}(n_k^{-\frac{2 β}{2 β+\boldsymbolΛ}});整体泛化误差与数据似然之差可表示为关于核心集权重\boldsymbol{w}和样本数n_k的闭式函数\boldsymbol{\Im}(\boldsymbol{w}, n_k)。在多个基准数据集上的实验显示,该方法相比随机采样后联邦学习显著提升性能,表明智能选样能有效增强模型表现。此外,在医学数据集上的实验表明,本方法优于其他基于子模优化的子集选择方法。
原文摘要 · Abstract (English)
In a distributed machine learning setting like Federated Learning where there are multiple clients involved which update their individual weights to a single central server, often training on the entire individual client's dataset for each client becomes cumbersome. To address this issue we propose $\methodprop$: a personalized coreset weighted federated learning setup where the training updates for each individual clients are forwarded to the central server based on only individual client coreset based representative data points instead of the entire client data. Through theoretical analysis we present how the average generalization error is minimax optimal up to logarithm bounds (upper bounded by $\mathcal{O}(n_k^{-\frac{2 β}{2 β+\boldsymbolΛ}} \log ^{2 δ^{\prime}}(n_k))$) and lower bounds of $\mathcal{O}(n_k^{-\frac{2 β}{2 β+\boldsymbolΛ}})$, and how the overall generalization error on the data likelihood differs from a vanilla Federated Learning setup as a closed form function ${\boldsymbol{\Im}}(\boldsymbol{w}, n_k)$ of the coreset weights $\boldsymbol{w}$ and coreset sample size $n_k$. Our experiments on different benchmark datasets based on a variety of recent personalized federated learning architectures show significant gains as compared to random sampling on the training data followed by federated learning, thereby indicating how intelligently selecting such training samples can help in performance. Additionally, through experiments on medical datasets our proposed method showcases some gains as compared to other submodular optimization based approaches used for subset selection on client's data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。