让语音识别模型在边缘设备上更省力,还能保持高精度。
Splitformer: An improved early-exit architecture for automatic speech recognition on edge devices
- 引入并行处理层,对下采样输入进行独立计算
- 在标准数据集上显著提升识别准确率,参数略增但推理时间不变
- 适合资源受限的实时语音识别场景
在边缘设备上进行语音识别时,动态调整模型计算负载至关重要,因设备资源有限且随时间变化。早期退出架构通过在中间层提前退出,减少计算量,是有效解决方案。然而,现有高效模型如Zipformer虽通过可变帧率分析降低运算量,却缺乏模块化设计,难以嵌入早期退出机制。本文提出在架构中引入并行层,对下采样后的输入进行独立处理,与标准处理路径协同工作。实验表明,该方法在标准基准测试上显著提升语音识别性能,仅略微增加模型参数量,且不增加推理时间。
原文摘要 · Abstract (English)
The ability to dynamically adjust the computational load of neural models during inference in a resource aware manner is crucial for on-device processing scenarios, characterised by limited and time-varying computational resources. Early-exit architectures represent an elegant and effective solution, since they can process the input with a subset of their layers, exiting at intermediate branches (the upmost layers are hence removed from the model). From a different perspective, for automatic speech recognition applications there are memory-efficient neural architectures that apply variable frame rate analysis, through downsampling/upsampling operations in the middle layers, reducing the overall number of operations and improving significantly the performance on well established benchmarks. One example is the Zipformer. However, these architectures lack the modularity necessary to inject early-exit branches. With the aim of improving the performance in early-exit models, we propose introducing parallel layers in the architecture that process downsampled versions of their inputs. % in conjunction with standard processing layers. We show that in this way the speech recognition performance on standard benchmarks significantly improve, at the cost of a small increase in the overall number of model parameters but without affecting the inference time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。