用提示词生成新候选，提升语音识别n-best结果的准确率。
ProGRes: Prompted Generative Rescoring on ASR n-Best
- 通过提示词驱动大模型动态生成新候选假设
- 在多个语音识别器上实现5%~25%的词错误率降低
- 适合需要高精度语音识别的场景
大型语言模型（LLMs）已证明可通过重新评分解码过程中的n-best候选假设，有效提升语音识别性能。然而，如何充分利用近期指令微调的生成式LLM进行候选重评分仍不明确。本文提出一种新方法，利用指令微调的LLM通过恰当提示动态扩展n-best语音识别候选集，生成新假设。具体地，提出一种零样本的ASR n-best重评分方法，结合置信度分数、LLM序列评分与提示驱动的假设生成。我们以Llama-3-Instruct、GPT-3.5 Turbo和GPT-4 Turbo作为提示生成器，使用Llama-3作为序列评分器。在不同语音识别器上评估该方法，观察到词错误率（WER）相对降低5%至25%。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have shown their ability to improve the performance of speech recognizers by effectively rescoring the n-best hypotheses generated during the beam search process. However, the best way to exploit recent generative instruction-tuned LLMs for hypothesis rescoring is still unclear. This paper proposes a novel method that uses instruction-tuned LLMs to dynamically expand the n-best speech recognition hypotheses with new hypotheses generated through appropriately-prompted LLMs. Specifically, we introduce a new zero-shot method for ASR n-best rescoring, which combines confidence scores, LLM sequence scoring, and prompt-based hypothesis generation. We compare Llama-3-Instruct, GPT-3.5 Turbo, and GPT-4 Turbo as prompt-based generators with Llama-3 as sequence scorer LLM. We evaluated our approach using different speech recognizers and observed significant relative improvement in the word error rate (WER) ranging from 5% to 25%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。