通过分解卷积核模拟更多客户端，缓解联邦学习中的数据异质性问题。
Extra Clients at No Extra Cost: Overcome Data Heterogeneity in Federated Learning with Filter Decomposition
- 将卷积核拆分为原子基和系数，实现原子级聚合
- 理论与实证均表明模型方差显著降低，准确率提升明显
- 支持灵活训练策略，适合个性化建模与高效通信场景
数据异质性是联邦学习（FL）的主要挑战之一，导致客户端间方差大、收敛慢。本文提出一种新方法：将联邦学习中的卷积滤波器分解为滤波器子空间元素（即滤波器原子）的线性组合。该技术将全局滤波器聚合转化为对滤波器原子及其系数的聚合。其核心优势在于，通过展开滤波器原子与系数加权和的乘积，数学上生成大量交叉项，有效模拟出众多潜在客户端，显著降低模型方差，这一结论得到理论分析与实验观察的双重验证。此外，该方法允许对滤波器原子与系数采用不同训练方案，实现高度自适应的模型个性化与通信效率优化。在基准数据集上的实验结果表明，滤波器分解技术显著提升了联邦学习方法的准确性，证实了其在应对数据异质性方面的有效性。
原文摘要 · Abstract (English)
Data heterogeneity is one of the major challenges in federated learning (FL), which results in substantial client variance and slow convergence. In this study, we propose a novel solution: decomposing a convolutional filter in FL into a linear combination of filter subspace elements, i.e., filter atoms. This simple technique transforms global filter aggregation in FL into aggregating filter atoms and their atom coefficients. The key advantage here involves mathematically generating numerous cross-terms by expanding the product of two weighted sums from filter atom and atom coefficient. These cross-terms effectively emulate many additional latent clients, significantly reducing model variance, which is validated by our theoretical analysis and empirical observation. Furthermore, our method permits different training schemes for filter atoms and atom coefficients for highly adaptive model personalization and communication efficiency. Empirical results on benchmark datasets demonstrate that our filter decomposition technique substantially improves the accuracy of FL methods, confirming its efficacy in addressing data heterogeneity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。