受大脑神经元启发,提出高效低功耗的新型人工神经元模型。
Threshold Neuron: A Brain-inspired Artificial Neuron for Efficient On-device Inference
- 模仿生物神经元阈值机制与兴奋抑制平衡设计新神经元。
- 在核级实现7.51至8.19倍功耗降低,面积节省3.89至4.33倍。
- 适合移动端和边缘设备部署,兼顾性能与能效。
在移动与边缘计算中,提升深度神经网络(DNNs)的计算效率仍是重大挑战。随着任务复杂度增加而算力受限,现有研究多聚焦于网络结构压缩与系统优化,却较少关注神经网络基本单元——神经元的改进。本文提出一种受生物神经元阈值机制与兴奋-抑制平衡启发的新型人工神经元模型:Threshold Neurons。该模型可构建与传统神经元相似的网络,显著降低硬件实现复杂度。大量实验验证其有效性:在核级实现7.51x至8.19x的功耗节省、3.89x至4.33x的面积节省,精度损失极小。FPGA实测显示系统级功耗降低2.52x,速度提升1.75x。源码将于发表后公开。
原文摘要 · Abstract (English)
Enhancing the computational efficiency of on-device Deep Neural Networks (DNNs) remains a significant challengein mobile and edge computing. As we aim to execute increasingly complex tasks with constrained computational resources, much of the research has focused on compressing neural network structures and optimizing systems. Although many studies have focused on compressing neural network structures and parameters or optimizing underlying systems, there has been limited attention on optimizing the fundamental building blocks of neural networks: the neurons. In this study, we deliberate on a simple but important research question: Can we design artificial neurons that offer greater efficiency than the traditional neuron paradigm? Inspired by the threshold mechanisms and the excitation-inhibition balance observed in biological neurons, we propose a novel artificial neuron model, Threshold Neurons. Using Threshold Neurons, we can construct neural networks similar to those with traditional artificial neurons, while significantly reducing hardware implementation complexity. Our extensive experiments validate the effectiveness of neural networks utilizing Threshold Neurons, achieving substantial power savings of 7.51x to 8.19x and area savings of 3.89x to 4.33x at the kernel level, with minimal loss in precision. Furthermore, FPGA-based implementations of these networks demonstrate 2.52x power savings and 1.75x speed enhancements at the system level. The source code will be made available upon publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。