arXiv:2607.04189cs.LG2026-07

通过频域过滤低频梯度噪声,提升联邦学习在数据异构下的收敛性。

SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity

论文配图:SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity
图 1 · 摘自论文原文
  • 从频域视角分析梯度偏移,发现异构主要体现在低频分量。
  • 抑制低频不一致梯度后,非独立同分布下准确率提升5.2%以上。
  • 支持快速近似实现,适合实际系统部署,通信开销几乎为零。

联邦学习受统计异构性严重挑战,非独立同分布(Non-IID)数据导致客户端梯度漂移,阻碍全局收敛。现有方法多在空间域进行梯度修正或正则化,却忽视了优化信号的内在频谱结构。本文从频域新视角重新审视客户端漂移,揭示关键的‘漂移频谱偏差’:客户端间梯度差异主要集中于低频分量,编码了各客户端的数据分布偏移;而高频分量反映细粒度特征,相对一致。受此启发,我们提出SpecGradFilter——一种统一的频谱梯度过滤框架,通过抑制不一致的低频信号来缓解异构性。关键发现是该方法具备通用性,不仅可通过精确的FFT截断实现,也可用高斯去趋势等空间近似方式高效达成。在CIFAR-10/100和Tiny-ImageNet等基准上的大量实验表明,在高度Non-IID设置下,SpecGradFilter显著提升性能,且通信开销可忽略,确立了鲁棒联邦优化的新范式。

原文摘要 · Abstract (English)

Federated Learning (FL) is fundamentally challenged by statistical heterogeneity, where non-identically distributed (non-IID) data induces client drift that severely hampers global convergence. While existing approaches attempt to mitigate this drift through spatial-domain gradient correction or regularization, they overlook the intrinsic spectral structure of optimization signals. In this work, we revisit client drift from a novel frequency-domain perspective and uncover a critical Spectral Bias of Drift: inter-client gradient divergence is predominantly concentrated in low-frequency components which encode client-specific distributional shifts, while high-frequency components representing fine-grained features remain relatively consistent. Motivated by this, we propose SpecGradFilter, a unified Spectral Gradient Filtering Framework that tames heterogeneity by suppressing discordant low-frequency signals. Crucially, we demonstrate that SpecGradFilter is a generalizable principle, effective not only via precise FFT-based truncation but also through spatial approximations like Gaussian detrending. Extensive experiments on benchmarks such as CIFAR-10/100 and Tiny-ImageNet demonstrate that SpecGradFilter significantly performs better performance in highly Non-IID settings with negligible communication overhead, establishing a new paradigm for robust federated optimization.

联邦学习梯度过滤频谱分析异构优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。