用光电器件自然衰减实现脉冲神经网络,提升能效并保持高精度。
Otters: An Energy-Efficient SpikingTransformer via Optical Time-to-First-Spike Encoding
- 利用光电器件的自然信号衰减代替传统计算,直接实现时间到首次脉冲编码
- 在7个GLUE数据集上达到顶尖准确率,能效比现有先进脉冲网络提升1.77倍
- 适合需要低功耗推理的边缘设备,如物联网和移动终端
脉冲神经网络(SNNs)具有高能效潜力,尤其是采用时间到首次脉冲(TTFS)编码时,每个神经元最多仅发射一个脉冲,极大提升了稀疏性。然而,这种能效优势常因推理中需计算时间衰减函数及其与突触权重的乘法操作而无法实现。本文提出颠覆性思路:将光电器件中的自然信号衰减这一‘硬件缺陷’重新利用为TTFS的核心计算机制。研究团队制备了定制化的氧化铟光电突触,验证其天然物理衰减可直接实现所需的时间函数。通过将器件的模拟输出视为突触权重与时间衰减的融合结果,所提出的光学脉冲神经网络(Otters)彻底省去了昂贵的数字运算。为在复杂架构如Transformer中应用该范式,研究引入一种新型量化神经网络到SNN的转换算法。软硬件协同设计使模型在七个GLUE基准数据集上达到当前最优准确率,并基于商用22nm工艺的能量测量分析,实现较之前领先SNN高达1.77倍的能效提升。本工作确立了将基础器件物理直接转化为强大计算原语的新范式,所有代码与数据开源。
原文摘要 · Abstract (English)
Spiking neural networks (SNNs) promise high energy efficiency, particularly with time-to-first-spike (TTFS) encoding, which maximizes sparsity by emitting at most one spike per neuron. However, such energy advantage is often unrealized because inference requires evaluating a temporal decay function and subsequent multiplication with the synaptic weights. This paper challenges this costly approach by repurposing a physical hardware `bug', namely, the natural signal decay in optoelectronic devices, as the core computation of TTFS. We fabricated a custom indium oxide optoelectronic synapse, showing how its natural physical decay directly implements the required temporal function. By treating the device's analog output as the fused product of the synaptic weight and temporal decay, optoelectronic synaptic TTFS (named Otters) eliminates these expensive digital operations. To use the Otters paradigm in complex architectures like the transformer, which are challenging to train directly due to the sparsity issue, we introduce a novel quantized neural network-to-SNN conversion algorithm. This complete hardware-software co-design enables our model to achieve state-of-the-art accuracy across seven GLUE benchmark datasets and demonstrates a 1.77$\times$ improvement in energy efficiency over previous leading SNNs, based on a comprehensive analysis of compute, data movement, and memory access costs using energy measurements from a commercial 22nm process. Our work thus establishes a new paradigm for energy-efficient SNNs, translating fundamental device physics directly into powerful computational primitives. All codes and data are open source.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。