Sorbet让变压器模型在神经形态硬件上高效运行，省电27倍。
Sorbet: A Neuromorphic Hardware-Compatible Transformer-Based Spiking Language Model
- 用新型移位式软最大值和位移幂归一化替代耗能操作
- 模型压缩后仍保持竞争力，相比BERT节省27.16倍能量
- 适合边缘设备部署，尤其看重能效的隐私场景
由于隐私等原因，边缘设备上部署语言模型的需求日益增长。这催生了面向资源受限设备的小型语言模型，其中能效至关重要。脉冲神经网络（SNN）因能效高而具有前景，已有工作尝试在SNN上实现基于变压器的模型。然而，软最大值和层归一化等关键操作难以在神经形态硬件上实现，许多早期工作选择跳过这些操作。为解决此问题，我们提出Sorbet，一种更适配神经形态硬件的基于变压器的脉冲语言模型。Sorbet引入了一种新颖的基于移位的软最大值（PTsoftmax）和位移幂归一化（BSPN），分别用于替代耗能的操作。通过知识蒸馏和模型量化，Sorbet实现了高度压缩的二值权重模型，在保持竞争性能的同时，相比BERT实现了27.16倍的能量节省。我们在GLUE基准和一系列消融实验中验证了Sorbet，证明其作为语言模型推理节能方案的潜力。代码已公开于https://github.com/Kaiwen-Tang/Sorbet。
原文摘要 · Abstract (English)
For reasons such as privacy, there are use cases for language models at the edge. This has given rise to small language models targeted for deployment in resource-constrained devices where energy efficiency is critical. Spiking neural networks (SNNs) offer a promising solution due to their energy efficiency, and there are already works on realizing transformer-based models on SNNs. However, key operations like softmax and layer normalization (LN) are difficult to implement on neuromorphic hardware, and many of these early works sidestepped them. To address these challenges, we introduce Sorbet, a transformer-based spiking language model that is more neuromorphic hardware-compatible. Sorbet incorporates a novel shifting-based softmax called PTsoftmax and a Bit Shifting PowerNorm (BSPN), both designed to replace the respective energy-intensive operations. By leveraging knowledge distillation and model quantization, Sorbet achieved a highly compressed binary weight model that maintains competitive performance while achieving $27.16\times$ energy savings compared to BERT. We validate Sorbet through extensive testing on the GLUE benchmark and a series of ablation studies, demonstrating its potential as an energy-efficient solution for language model inference. Our code is publicly available at \href{https://github.com/Kaiwen-Tang/Sorbet}{https://github.com/Kaiwen-Tang/Sorbet}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。