arXiv:2606.30499cs.LG2026-06

用随机网络蒸馏发现客户端相似性,实现无需预设的自动协作集群。

Discovering Collaboration from Novelty: Random Network Distillation for Clustered Federated Learning

论文配图:Discovering Collaboration from Novelty: Random Network Distillation for Clustered Federated Learning
图 1 · 摘自论文原文
  • 基于随机网络蒸馏生成客户端间相似度信号,无需共享原始数据。
  • 提前发现有意义的客户端分组,降低通信与计算开销。
  • 适用于大规模分布式系统,支持动态自组织协作结构。

联邦学习在非独立同分布数据下常因单一全局模型无法捕捉客户端分布多样性而表现不佳。聚类联邦学习通过为相似客户端群组训练专用模型缓解此问题,但现有方法常将聚类分配与主训练过程耦合,增加计算和通信成本。本文提出一种轻量级聚类方法:每个客户端在其本地数据上训练一个紧凑的随机网络蒸馏预测器,并利用预测误差作为新颖性信号,估计与其他客户端的相似度。该方法可在联邦训练前发现有意义的客户端分组,且不需共享原始数据或反复评估主模型。关键在于,运行时各客户端基于局部新颖性估计自发形成协作联邦,适合于无法预先确定聚类数量或协作结构的大规模自主分布式系统。通过解耦聚类与学习过程,该方法提供了一种任务无关、高效的非独立同分布数据下的自主协作机制。

原文摘要 · Abstract (English)

Federated Learning often suffers under non-independently and identically distributed data, where a single global model may fail to represent the diversity of client distributions. Clustered Federated Learning mitigates this issue by training specialized models for groups of similar clients, but existing approaches often couple cluster assignment with the main training loop, increasing computational and communication costs. We propose a lightweight clustering approach based on Random Network Distillation. Each client trains a compact Random Network Distillation predictor on its local data and uses its prediction error as a novelty signal to estimate similarity with other clients. This enables the discovery of meaningful client groups before federated training, without sharing raw data or repeatedly evaluating the main model. Crucially, the resulting federations emerge from local novelty estimates at runtime, making the method suitable for autonomous large-scale distributed systems where neither the number of clusters nor the collaboration structure can be specified a priori. Overall, by decoupling clustering from learning, the method provides a task-agnostic and efficient mechanism for autonomous collaboration under non-independently and identically distributed data.

联邦学习聚类协作自组织

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。