解决去中心化联邦多任务学习中的任务异构问题
ColNet: Collaborative Optimization in Decentralized Federated Multi-task Learning Systems
- 按模型与数据敏感度自适应聚类，分组优化
- 组内平均骨干网络，组间抗冲突聚合提升性能
- 适合任务差异大、需防攻击的分布式场景
联邦学习（FL）与多任务学习（MTL）的结合被用于应对客户端异构性，其中联邦多任务学习（FMTL）将每个客户端视为独立任务。然而，现有研究多聚焦于数据异构性（如非独立同分布数据），而忽视了任务异构性——即客户端解决根本不同的任务。此外，多数工作依赖由服务器管理的集中式设置，去中心化FMTL领域仍待探索。为此，本文提出ColNet框架，专为去中心化联邦环境中的异构任务设计。ColNet将模型分为骨干网络和任务特定头，基于模型与数据敏感度进行自适应聚类，形成任务一致的客户端组。组内骨干网络进行平均，组领导者执行抗冲突的跨组聚合。在多个数据集和联邦配置下，ColNet在标签与任务异构性下均优于对比方案，并对投毒攻击表现出鲁棒性。
原文摘要 · Abstract (English)
The integration of Federated Learning (FL) and Multi-Task Learning (MTL) has been explored to address client heterogeneity, with Federated Multi-Task Learning (FMTL) treating each client as a distinct task. However, most existing research focuses on data heterogeneity (e.g., addressing non-IID data) rather than task heterogeneity, where clients solve fundamentally different tasks. Additionally, much of the work relies on centralized settings with a server managing the federation, leaving the more challenging domain of decentralized FMTL largely unexplored. Thus, this work bridges this gap by proposing ColNet, a framework designed for heterogeneous tasks in decentralized federated environments. ColNet partitions models into a backbone and task-specific heads, and uses adaptive clustering based on model and data sensitivity to form task-coherent client groups. Backbones are averaged within groups, and group leaders perform hyper-conflict-averse cross-group aggregation. Across datasets and federations, ColNet outperforms competing schemes under label and task heterogeneity and shows robustness to poisoning attacks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。