通过稀疏化量化线性注意力模型，实现低功耗高效推理。
Event-Driven Language Models with Sparse Neural Activity for Neuromorphic Hardware
- 量化后引入可训练阈值稀疏激活，仅保留关键神经元
- 相比密集模型减少4倍有效运算量，性能几乎无损
- 适合事件驱动多核类脑硬件，吞吐提升37倍
基于Transformer的大语言模型（LLM）推理常受限于内存密集的键值缓存和二次方复杂度的注意力计算。状态空间模型（SSMs）通过线性注意力和固定大小的递归状态缓解此问题，但其大密度线性投影在量化后仍计算开销巨大。本文提出一种方法，在高度量化线性注意力模型中诱导稀疏神经活动，仅需极小性能损失。低于每投影可训练阈值（±Δ）的激活被置零，同时保留关键异常值，实现与密集模型相当的性能，且有效算术操作减少最多4倍。针对多核、多芯片类脑平台，事件驱动执行将非结构化稀疏转化为计算与通信层面的吞吐提升，这是GPU架构根本缺乏的能力。相比同规模Transformer模型在边缘GPU上的推理，预测可实现最高37倍的吞吐提升和16倍的功耗降低；相较于非稀疏基线，提升达5.4倍。这些结果表明，稀疏化量化线性注意力模型是部署LLM于事件驱动多核平台的天然选择。
原文摘要 · Abstract (English)
Inference with transformer-based large language models (LLMs) is often limited by the memory-bound KV cache and quadratic attention cost. State-space models (SSMs) mitigate this through linear attention and fixed-size recurrent states, but their large dense linear projections remain computationally expensive even after quantization. We introduce a method that induces sparse neural activity in heavily quantized linear-attention models with minimal performance loss. Activations below a per-projection trainable threshold ($\pm Δ$) are nullified while preserving crucial outliers, achieving comparable performance to dense models with up to 4$\times$ fewer effective arithmetic operations. Targeting a multi-core, multi-chip neuromorphic platform, where event-driven execution converts unstructured sparsity into throughput at both the compute and communication levels, a capability GPU architectures fundamentally lack, we project up to 37$\times$ higher throughput and 16$\times$ lower power versus edge GPU inference of a comparable transformer-based model, and up to 5.4$\times$ improvements over the non-sparsified baseline. These results position sparse, quantized linear-attention models as a natural fit for deploying LLMs on event-driven multi-core platforms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。