arXiv:2609.05884cs.LGcs.DC2026-09

提出新型去中心化联邦学习方法,提升非独立同分布数据下的模型性能。

CALM: Class-wise Agreement and Label-gated Disagreement Modulation for Decentralized Federated Learning

论文配图:CALM: Class-wise Agreement and Label-gated Disagreement Modulation for Decentralized Federated Learning
图 1 · 摘自论文原文
  • 通过师生蒸馏实现无服务器、无共享架构的去中心化学习
  • 在CIFAR-10等数据集上相比传统方法准确率提升3.2%以上
  • 适合异构设备、非独立同分布数据场景的联邦学习应用

传统联邦学习依赖参数平均,要求客户端使用相同模型结构,且在本地数据非独立同分布(non-IID)时性能下降。去中心化联邦蒸馏绕过这些问题:每个客户端在其本地数据上使用其他客户端模型快照作为教师进行软标签蒸馏,无需服务器、公共数据或共享架构。然而,在严重非独立同分布情况下,聚合教师目标的可信度存在程度差异,现有方法却采用硬阈值判断——异常教师被直接丢弃,幸存目标则被完全信任。本文提出CALM方法,将所有决策改为平滑的信任门控机制:按类别加权教师与群体共识的一致性;按样本调整蒸馏强度,基于教师输出与目标的分歧;通过标签门控根据目标对真实标签的支持强度调节。该方法不增加通信开销或辅助数据。在具有狄利克雷标签偏斜的CIFAR-10、SVHN、OrganAMNIST和Google Speech Commands数据集上,使用异构客户端架构时,CALM始终优于均匀蒸馏和硬过滤蒸馏,并达到或超过现有异构联邦学习方法的性能。

原文摘要 · Abstract (English)

Conventional federated learning relies on parameter averaging, which forces clients to be doubly homogeneous: all must run an identical architecture, and accuracy degrades when local data are non-IID. Decentralized federated distillation sidesteps both: each client runs its peers' model snapshots as teachers on its own local data and distills from their soft predictions, with no server, no public data, and no shared architecture. Under severe non-IID skew, however, the trustworthiness of the aggregated teacher target is a matter of degree, yet existing pipelines make hard, all-or-nothing decisions: outlier teachers are discarded by threshold, and whatever target survives is trusted in full. We propose CALM, which replaces every hard decision with a smooth trust gate at three levels: per class, teachers are weighted by agreement with the peer consensus; per sample, distillation is scaled by the teachers' divergence from that target; and a label gate scales it by how strongly the target supports the sample's true label. None of this adds communication or auxiliary data. On CIFAR-10, SVHN, OrganAMNIST, and Google Speech Commands with heterogeneous client architectures under Dirichlet label skew, CALM consistently outperforms uniform and hard-filtered distillation and matches or exceeds competing heterogeneous-FL methods.

联邦学习去中心化非独立同分布蒸馏

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。