通过帧级动态剪枝，显著提升语音识别解码效率。
FLToP CTC: Frame-Level Token Pruning via Relative Threshold for Efficient and Memory-Saving Decoding on Diverse Platforms
- 基于相对阈值在帧级别剪除低概率候选词
- 实测速度提升10.5倍，内存减少2.78倍
- 适合部署在资源受限设备如手机、嵌入式系统
基于CTC的语音识别系统在资源受限环境下面临计算与内存瓶颈。传统CTC解码器在系统中（如wav2vec2-large在L4 GPU上）消耗高达90%的处理时间，源于逐令牌的冗余计算。本文提出帧级令牌剪枝方法FLToP CTC，利用相对阈值概率动态剔除每帧中低置信度的候选词，显著降低计算与内存开销，同时保持可忽略的词错误率（WER）损失。在LibriSpeech数据集上，相比标准CTC解码器，FLToP CTC实现10.5倍的运行时加速和2.78倍的内存减少。该方法结构简单，可无缝集成至各类平台（CPU、GPU等）的CTC解码器中。本方法有效缓解了CTC解码瓶颈，为资源受限环境与实时应用提供可扩展方案，提升语音识别的实用性与普及性。
原文摘要 · Abstract (English)
CTC-based ASR systems face computational and memory bottlenecks in resource-limited environments. Traditional CTC decoders, requiring up to 90% of processing time in systems (e.g., wav2vec2-large on L4 GPUs), face inefficiencies due to exhaustive token-level operations. This paper introduces Frame Level Token Pruning for Connectionist Temporal Classification (FLToP CTC), a novel decoding algorithm that employs frame-level token pruning guided by a relative threshold probability. By dynamically eliminating low-probability tokens per frame, FLToP CTC reduces compute and memory demands while maintaining negligible WER degradation. On LibriSpeech, FLToP CTC achieves a 10.5x runtime speedup and 2.78x memory reduction versus standard CTC decoders. Its simplicity enables seamless integration into CTC decoders across platforms (CPUs, GPUs, etc.). FLToP CTC addresses CTC bottlenecks, offering scalability for resource-limited environments and realtime applications, enhancing speech recognition accessibility and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。