arXiv:2411.16442cs.LGcs.AI2024-11被引 6

TIFeD让极低资源设备也能参与联邦学习训练,用整数运算实现高效协作。

TIFeD: a Tiny Integer-based Federated learning algorithm with Direct feedback alignment

  • 全用整数运算,无需浮点支持,适合内存计算弱的设备
  • 支持全网络和单层模型训练,降低设备负载
  • 专为资源受限设备设计,适合边缘智能场景

在极低资源设备上直接训练机器学习模型是微型机器学习领域的新挑战。现有研究多聚焦于本地推理或在线模型适配,而将训练任务交由云端完成。联邦学习(FL)提供了一种分布式协同训练的可能,但当前主流算法难以在小型设备上运行。本文首次提出TIFeD:一种基于整数运算、采用直接反馈对齐(DFA)的微型联邦学习算法,完全由整数算术实现,专为内存、计算与能耗受限设备设计。除传统全网络训练模式外,还提出创新的单层实现方式,使每个设备仅训练神经网络的一部分,开启跨设备分布式学习新范式。实验验证了该方案的可行性和有效性。TIFeD的全网络与单层实现均已开源供学术界使用。

原文摘要 · Abstract (English)

Training machine and deep learning models directly on extremely resource-constrained devices is the next challenge in the field of tiny machine learning. The related literature in this field is very limited, since most of the solutions focus only on on-device inference or model adaptation through online learning, leaving the training to be carried out on external Cloud services. An interesting technological perspective is to exploit Federated Learning (FL), which allows multiple devices to collaboratively train a shared model in a distributed way. However, the main drawback of state-of-the-art FL algorithms is that they are not suitable for running on tiny devices. For the first time in the literature, in this paper we introduce TIFeD, a Tiny Integer-based Federated learning algorithm with Direct Feedback Alignment (DFA) entirely implemented by using an integer-only arithmetic and being specifically designed to operate on devices with limited resources in terms of memory, computation and energy. Besides the traditional full-network operating modality, in which each device of the FL setting trains the entire neural network on its own local data, we propose an innovative single-layer TIFeD implementation, which enables each device to train only a portion of the neural network model and opens the door to a new way of distributing the learning procedure across multiple devices. The experimental results show the feasibility and effectiveness of the proposed solution. The proposed TIFeD algorithm, with its full-network and single-layer implementations, is made available to the scientific community as a public repository.

联邦学习边缘计算整数运算微型模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。