arXiv:2504.09800cs.LGcs.AI2025-04被引 5

让不同任务的设备在联邦学习中共享通用知识,提升协作效率。

Multi-task Federated Learning with Encoder-Decoder Structure: Enabling Collaborative Learning Across Different Tasks

  • 用编码器-解码器结构分离任务特有与通用知识,实现跨任务共享。
  • 本地解码器逐步逼近全局解码器,达成跨任务知识融合。
  • 轻量模块化设计,适合多任务异构设备协同训练。

联邦学习因能在保障数据安全的前提下构建更优模型而受到广泛关注。然而,参与方常受限于模型结构或任务差异,无法协同学习。为此,本文提出一种基于编码器-解码器结构的多任务联邦学习框架(M-Fed)。利用当前主流模型普遍采用的编码器-解码器架构,通过传统联邦学习共享任务内知识,并从编码器中提取通用知识以实现跨任务共享。训练过程类似传统联邦学习,将本地解码器与全局解码器信息纳入损失函数,使本地解码器迭代更新并逐步逼近全局解码器,最终实现充分的跨任务知识融合。该方法轻量且模块化,在两个常用基准数据集上验证了其可行性,并优于传统方法,证明其在多任务联邦学习中的有效性。

原文摘要 · Abstract (English)

Federated learning has been extensively studied and applied due to its ability to ensure data security in distributed environments while building better models. However, clients participating in federated learning still face limitations, as clients with different structures or tasks cannot participate in learning together. In view of this, constructing a federated learning framework that allows collaboration between clients with different model structures and performing different tasks, enabling them to share valuable knowledge to enhance model efficiency, holds significant practical implications for the widespread application of federated learning. To achieve this goal, we propose a multi-task federated learning with encoder-decoder structure (M-Fed). Specifically, given the widespread adoption of the encoder-decoder architecture in current models, we leverage this structure to share intra-task knowledge through traditional federated learning methods and extract general knowledge from the encoder to achieve cross-task knowledge sharing. The training process is similar to traditional federated learning, and we incorporate local decoder and global decoder information into the loss function. The local decoder iteratively updates and gradually approaches the global decoder until sufficient cross-task knowledge sharing is achieved. Our method is lightweight and modular, demonstrating innovation compared to previous research. It enables clients performing different tasks to share general knowledge while maintaining the efficiency of traditional federated learning systems. We conducted experiments on two widely used benchmark datasets to verify the feasibility of M-Fed and compared it with traditional methods. The experimental results demonstrate the effectiveness of M-Fed in multi-task federated learning.

联邦学习多任务学习编码器-解码器知识共享

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。