arXiv:2409.05242cs.DCcs.AI2024-09被引 3

用频域变换压缩模型参数,显著降低联邦学习通信开销。

FedFT: Improving Communication Performance for Federated Learning with Frequency Space Transformation

  • 将模型参数通过DCT转换到频域,实现高效压缩传输
  • 相比原始方法通信开销降低5%至30%,精度保持甚至提升
  • 通用性强,兼容多种模型与联邦学习框架,适合资源受限场景

通信效率是联邦学习中的核心挑战。本文提出FedFT(联邦频域变换),利用离散余弦变换(DCT)将模型参数映射到频域,实现高效压缩并减少通信开销。该方法兼容多种现有联邦学习范式与神经网络架构,其线性特性避免了聚合时的重复变换。在四个数据集上与三种先进基线(FedAvg、FedProx、FedSim)对比表明,仅对参数差异进行频域表示,即可获得比完整模型频域表示更紧凑的表达,通信开销降低5%至30%,同时保持或提升模型精度。该方法有效应对数据隐私、系统互操作性与能效等分布式环境的关键问题。

原文摘要 · Abstract (English)

Communication efficiency is a widely recognised research problem in Federated Learning (FL), with recent work focused on developing techniques for efficient compression, distribution and aggregation of model parameters between clients and the server. Particularly within distributed systems, it is important to balance the need for computational cost and communication efficiency. However, existing methods are often constrained to specific applications and are less generalisable. In this paper, we introduce FedFT (federated frequency-space transformation), a simple yet effective methodology for communicating model parameters in a FL setting. FedFT uses Discrete Cosine Transform (DCT) to represent model parameters in frequency space, enabling efficient compression and reducing communication overhead. FedFT is compatible with various existing FL methodologies and neural architectures, and its linear property eliminates the need for multiple transformations during federated aggregation. This methodology is vital for distributed solutions, tackling essential challenges like data privacy, interoperability, and energy efficiency inherent to these environments. We demonstrate the generalisability of the FedFT methodology on four datasets using comparative studies with three state-of-the-art FL baselines (FedAvg, FedProx, FedSim). Our results demonstrate that using FedFT to represent the differences in model parameters between communication rounds in frequency space results in a more compact representation compared to representing the entire model in frequency space. This leads to a reduction in communication overhead, while keeping accuracy levels comparable and in some cases even improving it. Our results suggest that this reduction can range from 5% to 30% per client, depending on dataset.

联邦学习通信优化频域变换DCT

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。