通过稀疏原型压缩通信量，让边缘设备高效参与联邦学习。
TinyProto: Communication-Efficient Federated Learning with Sparse Prototypes in Resource-Constrained Environments
- 按类别分配特征维度，只传非零元素实现结构化稀疏。
- 相比现有方法通信量降低4倍，性能保持不变。
- 无需客户端额外计算，适合异构边缘设备部署。
联邦学习在资源受限环境中的通信效率仍是关键挑战。虽然基于原型的联邦学习通过共享类别原型（即倒数第二层的均值激活）而非模型参数来降低通信开销，但其效率会随特征维度和类别数量增加而下降。我们提出TinyProto，通过类别级原型稀疏化（CPS）和自适应原型缩放解决这一问题。CPS通过为每个类别分配特定特征维度，仅传输非零元素，实现结构化稀疏；自适应缩放则根据类别分布动态调整原型。实验表明，TinyProto相比现有方法通信成本最高降低4倍，同时保持性能。此外，该方法无需客户端额外计算开销，支持异构架构，特别适用于资源受限的异构联邦学习场景。
原文摘要 · Abstract (English)
Communication efficiency in federated learning (FL) remains a critical challenge for resource-constrained environments. While prototype-based FL reduces communication overhead by sharing class prototypes-mean activations in the penultimate layer-instead of model parameters, its efficiency decreases with larger feature dimensions and class counts. We propose TinyProto, which addresses these limitations through Class-wise Prototype Sparsification (CPS) and adaptive prototype scaling. CPS enables structured sparsity by allocating specific dimensions to class prototypes and transmitting only non-zero elements, while adaptive scaling adjusts prototypes based on class distributions. Our experiments show TinyProto reduces communication costs by up to 4x compared to existing methods while maintaining performance. Beyond its communication efficiency, TinyProto offers crucial advantages: achieving compression without client-side computational overhead and supporting heterogeneous architectures, making it ideal for resource-constrained heterogeneous FL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。