通过谱几何方法识别并过滤联邦学习中的恶意更新。
SpectralKrum: A Spectral-Geometric Defense Against Byzantine Attacks in Federated Learning
- 利用历史聚合结果估计良性更新的低维流形,投影新更新到该子空间。
- 在压缩坐标中用Krum选择,并以数据驱动阈值剔除异常能量更新。
- 无需额外数据,适用于非独立同分布场景,适合防御定向攻击。
联邦学习将模型训练分散到本地保存数据的客户端,但这种架构存在根本性漏洞:拜占庭客户端可注入任意损坏的更新,从而破坏或操控全局模型。尽管鲁棒聚合方法(如Krum、Bulyan和坐标级防御)在理想假设下具备理论保障,但在客户端数据分布异质(非独立同分布)且攻击者能观测或近似防御机制时,其效果显著下降。本文提出SpectralKrum,融合谱子空间估计与几何邻域选择。核心思想是:尽管各客户端存在异质性,良性优化轨迹仍集中在可从历史聚合中估计的低维流形上。SpectralKrum将输入更新投影至该学习子空间,在压缩坐标中应用Krum选择,并过滤正交残差能量超过数据驱动阈值的候选。该方法无需辅助数据,完全基于模型更新运行,且保持联邦学习隐私特性。我们在CIFAR-10上使用狄利克雷分布的非独立同分布划分(alpha = 0.1)评估SpectralKrum,覆盖七种攻击场景,实验涵盖超过56,000次训练轮次。结果表明,SpectralKrum在对抗方向性及子空间感知攻击(如adaptive-steer、buffer-drift)时表现良好,但在标签翻转和极小极大攻击下优势有限,因恶意更新在谱上难以与良性更新区分。
原文摘要 · Abstract (English)
Federated Learning (FL) distributes model training across clients who retain their data locally, but this architecture exposes a fundamental vulnerability: Byzantine clients can inject arbitrarily corrupted updates that degrade or subvert the global model. While robust aggregation methods (including Krum, Bulyan, and coordinate-wise defenses) offer theoretical guarantees under idealized assumptions, their effectiveness erodes substantially when client data distributions are heterogeneous (non-IID) and adversaries can observe or approximate the defense mechanism. This paper introduces SpectralKrum, a defense that fuses spectral subspace estimation with geometric neighbor-based selection. The core insight is that benign optimization trajectories, despite per-client heterogeneity, concentrate near a low-dimensional manifold that can be estimated from historical aggregates. SpectralKrum projects incoming updates into this learned subspace, applies Krum selection in compressed coordinates, and filters candidates whose orthogonal residual energy exceeds a data-driven threshold. The method requires no auxiliary data, operates entirely on model updates, and preserves FL privacy properties. We evaluate SpectralKrum against eight robust baselines across seven attack scenarios on CIFAR-10 with Dirichlet-distributed non-IID partitions (alpha = 0.1). Experiments spanning over 56,000 training rounds show that SpectralKrum is competitive against directional and subspace-aware attacks (adaptive-steer, buffer-drift), but offers limited advantage under label-flip and min-max attacks where malicious updates remain spectrally indistinguishable from benign ones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。