用脑启发的超维度计算,让无监督联邦学习更高效更抗噪。
FedUHD: Unsupervised Federated Learning using Hyperdimensional Computing
- 基于超维度计算框架,客户端去除非独立同分布数据中的噪声样本。
- 服务器端加权聚合提升模型精度,平均准确率高出15.50%。
- 训练速度提升173.6倍,通信开销降低271倍,适合边缘设备部署。
无监督联邦学习(UFL)作为一种保护隐私的去中心化机器学习方法,无需人工标注数据,但面临三大挑战:设备间数据非独立同分布(non-iid)、边缘端计算与通信成本高、易受通信噪声影响。现有方法依赖深度神经网络(NN),带来巨大计算与通信开销。本文提出首个基于超维度计算(HDC)的无监督联邦学习框架FedUHD。HDC具有轻量训练、小模型尺寸和强抗噪性。FedUHD在客户端引入基于kNN的聚类超向量剔除方法,消除非iid数据中的有害离群点;在服务器端设计加权HDC聚合策略,缓解客户端数据分布不均问题。实验表明,相比最先进的基于NN的UFL方法,FedUHD在训练上实现最高173.6倍加速、612.7倍能效提升,通信开销降低271倍,平均准确率提升15.50%,且对各类噪声具有更强鲁棒性。
原文摘要 · Abstract (English)
Unsupervised federated learning (UFL) has gained attention as a privacy-preserving, decentralized machine learning approach that eliminates the need for labor-intensive data labeling. However, UFL faces several challenges in practical applications: (1) non-independent and identically distributed (non-iid) data distribution across devices, (2) expensive computational and communication costs at the edge, and (3) vulnerability to communication noise. Previous UFL approaches have relied on deep neural networks (NN), which introduce substantial overhead in both computation and communication. In this paper, we propose FedUHD, the first UFL framework based on Hyperdimensional Computing (HDC). HDC is a brain-inspired computing scheme with lightweight training and inference operations, much smaller model size, and robustness to communication noise. FedUHD introduces two novel HDC-based designs to improve UFL performance. On the client side, a kNN-based cluster hypervector removal method addresses non-iid data samples by eliminating detrimental outliers. On the server side, a weighted HDC aggregation technique balances the non-iid data distribution across clients. Our experiments demonstrate that FedUHD achieves up to 173.6x and 612.7x better speedup and energy efficiency, respectively, in training, up to 271x lower communication cost, and 15.50% higher accuracy on average across diverse settings, along with superior robustness to various types of noise compared to state-of-the-art NN-based UFL approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。