arXiv:2505.04223cs.LGcs.AI2025-05

FRAIN让分布式训练更快更稳,解决慢设备和恶意节点干扰问题。

FRAIN to Train: A Fast-and-Reliable Solution for Decentralized Federated Learning

  • 引入FastSync机制,新人可快速逼近全局模型,无需重播历史版本。
  • 用SLERP融合参数,保持模型方向一致,减少分歧更新的破坏性影响。
  • 在非独立同分布数据、延迟网络和恶意节点环境下表现优于现有方法。

联邦学习(FL)可在保护数据本地性的同时实现跨分布式客户端的协同建模。尽管FedAvg采用同步轮次进行全局模型平均,但慢速设备会拖累整体进度。异步联邦学习(如FedAsync)通过持续整合客户端更新缓解了这一问题,但原始实现易受非独立同分布(non-IID)数据与过时贡献的影响,导致客户端漂移。部分基于区块链的方案(如BRAIN)虽通过更新加权或评分抵抗恶意或偏差提议,但在严重数据异质性或高过时率下性能仍下降,且无聚合器架构带来了新的同步开销。本文提出快速可靠人工智能网络FRAIN,通过两项关键设计克服上述局限:首先,FastSync策略消除重播旧模型版本的需求,使新加入者与低频参与者能高效近似全局模型;其次,在参数融合中采用球面线性插值(SLERP),保持模型方向一致性,减轻分歧训练带来的破坏性干扰。在CNN图像分类与Transformer语言模型上的实验表明,FRAIN在非独立同分布数据、存在延迟的网络环境及恶意节点条件下,收敛更稳定、鲁棒性更强,显著优于FedAvg、FedAsync与BRAIN。

原文摘要 · Abstract (English)

Federated learning (FL) enables collaborative model training across distributed clients while preserving data locality. Although FedAvg pioneered synchronous rounds for global model averaging, slower devices can delay collective progress. Asynchronous FL (e.g., FedAsync) addresses stragglers by continuously integrating client updates, yet naive implementations risk client drift due to non-IID data and stale contributions. Some Blockchain-based FL approaches (e.g., BRAIN) employ robust weighting or scoring of updates to resist malicious or misaligned proposals. However, performance drops can still persist under severe data heterogeneity or high staleness, and synchronization overhead has emerged as a new concern due to its aggregator-free architectures. We introduce Fast-and-Reliable AI Network, FRAIN, a new asynchronous FL method that mitigates these limitations by incorporating two key ideas. First, our FastSync strategy eliminates the need to replay past model versions, enabling newcomers and infrequent participants to efficiently approximate the global model. Second, we adopt spherical linear interpolation (SLERP) when merging parameters, preserving models' directions and alleviating destructive interference from divergent local training. Experiments with a CNN image-classification model and a Transformer-based language model demonstrate that FRAIN achieves more stable and robust convergence than FedAvg, FedAsync, and BRAIN, especially under harsh environments: non-IID data distributions, networks that experience delays and require frequent re-synchronization, and the presence of malicious nodes.

联邦学习异步训练模型融合安全协同

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。