改进Whisper对印度语的识别准确率与速度
Enhancing Whisper's Accuracy and Speed for Indian Languages through Prompt-Tuning and Tokenization
- 用语言家族信息做提示调优，提升相似语言识别准确率
- 新分词器减少生成标记数，显著加快推理速度
- 适合需要高效低资源语音识别的场景
自动语音识别近年来因Whisper等大型基础模型取得显著进展。然而，这些模型在低资源语言（如印度语）上表现不佳。本文提出两种新方法以提升Whisper在印度语中的多语言语音识别性能：一是利用语言家族信息进行提示调优，增强语言相近时的识别准确率；二是引入新型分词器，减少生成标记数量，从而加速Whisper的推理速度。大量实验表明，该分词器显著降低推理时间，而提示调优在Whisper的Small、Medium和Large模型上均有效提升准确率。二者结合实现了最优词错误率（WER）与推理速度之间的良好平衡。
原文摘要 · Abstract (English)
Automatic speech recognition has recently seen a significant advancement with large foundational models such as Whisper. However, these models often struggle to perform well in low-resource languages, such as Indian languages. This paper explores two novel approaches to enhance Whisper's multilingual speech recognition performance in Indian languages. First, we propose prompt-tuning with language family information, which enhances Whisper's accuracy in linguistically similar languages. Second, we introduce a novel tokenizer that reduces the number of generated tokens, thereby accelerating Whisper's inference speed. Our extensive experiments demonstrate that the tokenizer significantly reduces inference time, while prompt-tuning enhances accuracy across various Whisper model sizes, including Small, Medium, and Large. Together, these techniques achieve a balance between optimal WER and inference speed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。