用类脑芯片实现无矩阵乘法的低功耗大模型推理。
Neuromorphic Principles for Efficient Large Language Models on Intel Loihi 2
- 摒弃传统矩阵乘法，基于事件驱动与状态化计算设计新架构。
- 370万参数模型量化后零精度损失，推理速度提升3倍、能耗减半。
- 适合边缘设备上需要快速生成长文本的高效推理场景。
大型语言模型（LLMs）性能卓越但能耗巨大。本文提出一种适配英特尔类脑芯片Loihi 2的无矩阵乘法（MatMul-free）LLM架构，充分利用其低精度、事件驱动计算和状态化处理能力。在GPU上的硬件感知量化模型表明，370万参数的无矩阵乘法模型可实现零精度损失的量化。初步结果显示，相比边缘GPU上的基于Transformer的LLM，该方案可实现高达3倍的吞吐量提升，能耗降低2倍，且具有更优的扩展性。进一步硬件优化将带来更高吞吐与更低能耗。结果表明类脑硬件在高效推理方面潜力巨大，为快速、低成本生成复杂长文本的推理模型开辟了道路。
原文摘要 · Abstract (English)
Large language models (LLMs) deliver impressive performance but require large amounts of energy. In this work, we present a MatMul-free LLM architecture adapted for Intel's neuromorphic processor, Loihi 2. Our approach leverages Loihi 2's support for low-precision, event-driven computation and stateful processing. Our hardware-aware quantized model on GPU demonstrates that a 370M parameter MatMul-free model can be quantized with no accuracy loss. Based on preliminary results, we report up to 3x higher throughput with 2x less energy, compared to transformer-based LLMs on an edge GPU, with significantly better scaling. Further hardware optimizations will increase throughput and decrease energy consumption. These results show the potential of neuromorphic hardware for efficient inference and pave the way for efficient reasoning models capable of generating complex, long-form text rapidly and cost-effectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。