用微调与GELU适配器提升多语言语音识别准确率
Improving Code Switching with Supervised Fine Tuning and GELU Adapters
- 分语言令牌化方法让模型更好处理多语混杂语音
- 在ASCEND等数据集上错误率降至9.4%以下
- 适合需要高精度多语言语音识别的场景
目前可用的多语言语音识别数据集极少，导致自动语音识别（ASR）难以有效利用大量单语数据和预训练模型。本文基于OpenAI开源的Whisper模型（在68万小时音频上预训练），提出两种改进方法：第一，采用分语言令牌化策略，分别对不同语言文本进行独立分词；第二，在编码器中引入基于GELU的适配器进行微调。结合两者后，总混合错误率（MER）在ASCEND数据集上降至9.4%，SEAME devman为6%，SEAME devsge为9.7%，优于现有最先进方法。
原文摘要 · Abstract (English)
There are few code switching datasets, labeled or unlabled, that exist today. As a result, ASR requires new methods to utilize the vast monolingual data and models that exist. This paper uses OpenAI's open source ASR model, Whisper, which has been pre-trained on 680K hours of audio to perform monolingual ASR tasks. In Part 1, this paper examines how exploiting Whisper's monolingual ability to individually tokenize training text, called "Switching Tokenizers Method", improves transcription accuracy. In Part 2, we combine the Switching Tokenizers Method from part 1 and train a GELU based adapter on the encoder. These two methods reduced Total Mixed Error Rate (MER) to 9.4% for the ASCEND dataset, 6% for SEAME devman and 9.7% for SEAME devsge, outperforming current SoTA methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。