arXiv:2507.07396cs.MMcs.LG2025-07中稿 · TNNLS被引 2

用新型脉冲网络实现高效语音识别,性能接近传统模型。

IML-Spikeformer: Input-aware Multi-Level Spiking Transformer for Speech Processing

  • 设计输入感知的多层级脉冲机制,单步模拟多时步放电。
  • 在AiShell-1和Librispeech-960上分别达到6.0%和3.4%错误率。
  • 适合追求低功耗语音处理的科研与工程应用。

受生物神经机制启发,脉冲神经网络(SNN)是具能效优势的类脑计算范式。尽管有效,现有SNN在大规模语音任务中表现仍不理想,主要受限于训练时多时步脉冲放电带来的高计算开销,以及缺乏适配语音任务的大规模架构。为此,本文提出面向大规模语音处理的输入感知多层级脉冲变换器(IML-Spikeformer)。核心为输入感知多层级脉冲(IMLS)机制,通过自适应阈值策略在单个时间步内模拟多时步脉冲放电。进一步引入可重参数化脉冲自注意力(RepSSA)与分层衰减掩码(HDM),形成HD-RepSSA模块,提升注意力精度并建模语音信号的多尺度时序依赖。实验表明,IML-Spikeformer在AiShell-1上达到6.0%词错误率,在Librispeech-960上达3.4%,性能媲美传统ANN Transformer,同时理论推理能耗分别降低4.64倍和4.32倍。该工作推动了大规模语音处理中可扩展SNN架构的发展。源码与模型权重已开源。

原文摘要 · Abstract (English)

Spiking Neural Networks (SNNs), inspired by biological neural mechanisms, represent a promising neuromorphic computing paradigm that offers energy-efficient alternatives to traditional Artificial Neural Networks (ANNs). Despite proven effectiveness, SNN architectures have struggled to achieve competitive performance on large-scale speech processing tasks. Two key challenges hinder progress: (1) the high computational overhead during training caused by multi-timestep spike firing, and (2) the absence of large-scale SNN architectures tailored to speech processing tasks. To overcome the issues, we introduce Input-aware Multi-Level Spikeformer, i.e. IML-Spikeformer, a spiking Transformer architecture specifically designed for large-scale speech processing. Central to our design is the Input-aware Multi-Level Spike (IMLS) mechanism, which simulates multi-timestep spike firing within a single timestep using an adaptive, input-aware thresholding scheme. IML-Spikeformer further integrates a Re-parameterized Spiking Self-Attention (RepSSA) module with a Hierarchical Decay Mask (HDM), forming the HD-RepSSA module. This module enhances the precision of attention maps and enables modeling of multi-scale temporal dependencies in speech signals. Experiments demonstrate that IML-Spikeformer achieves word error rates of 6.0\% on AiShell-1 and 3.4\% on Librispeech-960, comparable to conventional ANN transformers while reducing theoretical inference energy consumption by 4.64$\times$ and 4.32$\times$ respectively. IML-Spikeformer marks an advance of scalable SNN architectures for large-scale speech processing in both task performance and energy efficiency. Our source code and model checkpoints are publicly available at github.com/Pooookeman/IML-Spikeformer.

脉冲神经网络语音识别低功耗计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。