预训练语言模型的Transformer可作语音识别编码器，提升识别准确率。
Are Transformers in Pre-trained LM A Good ASR Encoder? An Empirical Study
- 用预训练语言模型中的Transformer做语音识别编码器，利用其文本语义能力。
- 在多种语音识别任务中，字符错误率和词错误率均显著降低。
- 特别适合需要深层语义理解的语音识别场景，如口语理解。
本研究探讨了预训练语言模型（PLM）中的Transformer作为自动语音识别（ASR）编码器的有效性。尽管这些Transformer最初仅在文本语料上训练，但实验表明它们仍具备从输入序列中提取有效特征的能力。这种能力可迁移至语音数据，从而增强ASR的声学建模性能。通过严格的实证分析，结果发现：在多个ASR任务中引入来自预训练语言模型的Transformer后，字符错误率（CER）和词错误率（WER）均有显著下降；尤其在需要深度语义理解的任务中，性能提升更为明显。这表明，可利用预训练Transformer内嵌的语义理解能力，显著提升现有ASR系统的识别能力。
原文摘要 · Abstract (English)
In this study, we delve into the efficacy of transformers within pre-trained language models (PLMs) when repurposed as encoders for Automatic Speech Recognition (ASR). Our underlying hypothesis posits that, despite being initially trained on text-based corpora, these transformers possess a remarkable capacity to extract effective features from the input sequence. This inherent capability, we argue, is transferrable to speech data, thereby augmenting the acoustic modeling ability of ASR. Through rigorous empirical analysis, our findings reveal a notable improvement in Character Error Rate (CER) and Word Error Rate (WER) across diverse ASR tasks when transformers from pre-trained LMs are incorporated. Particularly, they serve as an advantageous starting point for initializing ASR encoders. Furthermore, we uncover that these transformers, when integrated into a well-established ASR encoder, can significantly boost performance, especially in scenarios where profound semantic comprehension is pivotal. This underscores the potential of leveraging the semantic prowess embedded within pre-trained transformers to advance ASR systems' capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。