通过外部参考原型对齐,提升联邦学习通信效率与模型泛化能力
RefProtoFL: Communication-Efficient Federated Learning via External-Referenced Prototype Alignment
- 分离私有主干与轻量共享适配器,仅传输适配器参数
- 在相同通信轮次下,准确率优于现有原型联邦方法
- 适合数据异构、带宽受限的边缘联邦学习场景
联邦学习(FL)可在不共享原始数据的情况下实现协同建模,但受限于通信带宽和客户端数据分布异构。基于原型的联邦学习通过交换类别特征原型而非完整模型参数缓解此问题,但现有方法在严苛通信约束下仍存在泛化性能不足。本文提出RefProtoFL框架,结合外部参考原型对齐(ERPA)以增强表示一致性,以及自适应概率更新丢弃(APUD)以提升通信效率。具体地,将模型拆分为私有主干与轻量共享适配器,仅传输适配器参数;为降低上行开销,APUD采用基于重要性的Top-K稀疏化,仅上传显著的适配器更新。为解决异构客户端间表示不一致问题,ERPA利用服务器端持有的一小部分公开数据构建外部参考原型作为共享语义锚点。对于公共数据覆盖的类别,客户端直接对齐本地表示至公共诱导原型;未覆盖类别则通过加权平均聚合全局参考原型进行对齐。在标准基准上的大量实验表明,RefProtoFL在相同通信轮次下分类准确率优于当前最优原型类联邦学习方法。
原文摘要 · Abstract (English)
Federated learning (FL) enables collaborative model training without sharing raw data in edge environments, but is constrained by limited communication bandwidth and heterogeneous client data distributions. Prototype-based FL mitigates this issue by exchanging class-wise feature prototypes instead of full model parameters; however, existing methods still suffer from suboptimal generalization under severe communication constraints. In this paper, we propose RefProtoFL, a communication-efficient FL framework that integrates External-Referenced Prototype Alignment (ERPA) for representation consistency with Adaptive Probabilistic Update Dropping (APUD) for communication efficiency. Specifically, we decompose the model into a private backbone and a lightweight shared adapter, and restrict federated communication to the adapter parameters only. To further reduce uplink cost, APUD performs magnitude-aware Top-K sparsification, transmitting only the most significant adapter updates for server-side aggregation. To address representation inconsistency across heterogeneous clients, ERPA leverages a small server-held public dataset to construct external reference prototypes that serve as shared semantic anchors. For classes covered by public data, clients directly align local representations to public-induced prototypes, whereas for uncovered classes, alignment relies on server-aggregated global reference prototypes via weighted averaging. Extensive experiments on standard benchmarks demonstrate that RefProtoFL attains higher classification accuracy than state-of-the-art prototype-based FL baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。