快速转换大模型为脉冲神经网络，显著降低能耗与延迟
FAS: Fast ANN-SNN Conversion for Spiking Large Language Models
- 分两阶段转换：全参数微调+粗到细校准
- 8个时间步达OPT-7B高3%准确率，能效降96.63%
- 适合低功耗部署的LLM应用
脉冲大语言模型在多种场景下展现出对传统大模型的良好替代性。现有构建脉冲LLM的方法——直接训练或ANN-SNN转换——常面临性能下降和计算成本较高的问题。为此，我们提出一种新型快速ANN-SNN转换策略（FAS），分两个阶段将大模型转化为脉冲模型。第一阶段采用预训练模型的全参数微调，无需从头训练；第二阶段引入粗到细校准方法，降低转换误差，提升精度。在四个不同规模的大模型上，针对语言和视觉-语言任务的实验表明，FAS可实现当前最优性能，同时显著降低推理延迟与计算开销。特别地，FAS仅需8个时间步即可达到比OPT-7B高3%的准确率，且能量消耗降低96.63%。源代码已公开于https://github.com/lc783/FAS。
原文摘要 · Abstract (English)
Spiking Large Language Models have been shown as a good alternative to LLMs in various scenarios. Existing methods for creating Spiking LLMs, i.e., direct training and ANN-SNN conversion, often suffer from performance degradation and relatively high computational costs. To address these issues, we propose a novel Fast ANN-SNN conversion strategy (FAS) that transforms LLMs into spiking LLMs in two stages. The first stage employs a full-parameter fine-tuning of pre-trained models, so it does not need any direct training from scratch. The second stage introduces a coarse-to-fine calibration method to reduce conversion errors and improve accuracy. Experiments on both language and vision-language tasks across four different scales of LLMs demonstrate that FAS can achieve state-of-the-art performance yet with significantly reduced inference latency and computational costs. Notably, FAS only takes eight timesteps to achieve an accuracy of 3\% higher than that of the OPT-7B model, while reducing energy consumption by 96.63\%. The source code is available at https://github.com/lc783/FAS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。