用大模型语义特征实现一次性的分类感知客户端聚类，提升异构联邦学习效果。
FMCL: Class-Aware Client Clustering with Foundation Model Representations for Heterogeneous Federated Learning
- 用冻结的大模型提取客户端的类别级嵌入原型，构建语义签名。
- 在非同分布数据下，相比现有方法提升性能并更稳定聚类。
- 无需训练时通信，适用于多种下游模型，适合异构联邦场景。
联邦学习可在不共享原始数据的情况下跨分布式客户端协同训练，但在统计异构性下性能下降。聚类联邦学习通过分组相似客户端并为每组训练独立模型来缓解该问题。然而，现有聚类策略多依赖原始数据统计、模型参数或启发式相似度度量，难以捕捉异构领域间的类别级语义结构，且常需迭代协调。本文提出FMCL，一种一次性、分类感知的客户端聚类框架，利用大模型表示构建语义客户端签名。通过冻结的大模型计算每个客户端的类别级嵌入原型，并以余弦距离衡量其类感知表示的相似性。聚类在训练前一次性完成，不引入额外通信开销，且对下游模型架构无感。在多个异构基准上的实验表明，FMCL在非同分布数据划分下显著提升联邦学习性能，并展现出更稳定的聚类行为。
原文摘要 · Abstract (English)
Federated Learning (FL) enables collaborative model training across distributed clients without sharing raw data, yet its performance deteriorates under statistical heterogeneity. Clustered Federated Learning addresses this challenge by grouping similar clients and training separate models per cluster. However, existing clustering strategies often rely on raw data statistics, model parameters, or heuristic similarity measures that fail to capture class-level semantic structure across heterogeneous domains and frequently require iterative coordination. We propose FMCL, a one-shot, class-aware client clustering framework that leverages foundation model representations to construct semantic client signatures. Using a frozen foundation model, FMCL computes class-level embedding prototypes for each client and measures similarity via cosine distance between their class-aware representations. Clustering is performed once prior to training, introducing no additional communication during federated optimization and remaining agnostic to the downstream model architecture. Extensive experiments across heterogeneous benchmarks demonstrate that FMCL improves federated performance and yields more stable clustering behavior compared to existing clustering-based methods under non-identically distributed data partitioning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。