无需额外模型，用词频地图加速语音识别推理
Model-free Speculative Decoding for Transformer-based ASR with Token Map Drafting
- 用领域数据预构建的n-gram词频图替代专用草稿模型
- 在CI-AVSR和内部数据集上提速1.27~1.37倍，准确率不变
- 适合无GPU的设备部署，尤其适用于低困惑度场景
基于Transformer的端到端语音识别系统（如Whisper）虽具备高精度和鲁棒性，但自回归解码计算开销大，限制了在无硬件加速器的CPU设备上的部署。推测解码（SD）通过小模型提出候选词再由主模型验证来缓解此问题，但依赖额外模型难以在资源受限设备使用。本文提出「词元映射草稿」（Token Map Drafting），一种无需训练草稿模型的模型无关方法：利用领域特定训练数据预先构建的n-gram词元映射，实现高效推测解码，几乎无额外开销。实验表明，在结构化、低困惑度领域中，该方法显著加速推理，且不损失识别准确率。在CI-AVSR数据集上实现1.27倍加速，在内部数据集上达1.37倍；相比基于蒸馏的基线，在CPU上提升10%的解码速度，充分展现其在本地化语音识别应用中的有效性。
原文摘要 · Abstract (English)
End-to-end automatic speech recognition (ASR) systems based on transformer architectures, such as Whisper, offer high transcription accuracy and robustness. However, their autoregressive decoding is computationally expensive, hence limiting deployment on CPU-based and resource-constrained devices. Speculative decoding (SD) mitigates this issue by using a smaller draft model to propose candidate tokens, which are then verified by the main model. However, this approach is impractical for devices lacking hardware accelerators like GPUs. To address this, we propose \emph{Token Map Drafting}, a model-free SD technique that eliminates the need for a separate draft model. Instead, we leverage a precomputed n-gram token map derived from domain-specific training data, enabling efficient speculative decoding with minimal overhead. Our method significantly accelerates ASR inference in structured, low-perplexity domains without sacrificing transcription accuracy. Experimental results demonstrate decoding speed-ups of $1.27\times$ on the CI-AVSR dataset and $1.37\times$ on our internal dataset without degrading recognition accuracy. Additionally, our approach achieves a $10\%$ absolute improvement in decoding speed over the Distill-spec baseline running on CPU, highlighting its effectiveness for on-device ASR applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。