用统一罗马化转写实现多语言语音识别,少数据也能媲美顶尖模型。
LAMA-UT: Language Agnostic Multilingual ASR through Orthography Unification and Language-Specific Transliteration
- 先统一转写为罗马字母,再转回各语言特定形式。
- 仅用0.1%数据训练,错误率比Whisper低45%。
- 无需语言专属模块,适合新语言快速接入。
构建公平表现的通用多语言自动语音识别模型长期面临挑战。本文提出无需语言专属模块的统一多语言语音识别框架LAMA-UT,通过统一转写与语言特异性音译实现。首先,利用通用转写生成器将不同语言的书写特征统一为罗马化形式,捕捉共通语音特征;其次,通过通用转换器将统一转写结果还原为各语言特定形式。实验表明,该方法在大规模多语言识别中有效:相比Whisper,相对错误率降低45%,且仅使用Whisper 0.1%的数据量即可达到MMS水平性能。此外,该框架不依赖任何语言专属模块,仍能与使用额外语言词典和语言模型的零样本方法性能相当,有望成为面向未见语言的可扩展多语言语音识别基石。
原文摘要 · Abstract (English)
Building a universal multilingual automatic speech recognition (ASR) model that performs equitably across languages has long been a challenge due to its inherent difficulties. To address this task we introduce a Language-Agnostic Multilingual ASR pipeline through orthography Unification and language-specific Transliteration (LAMA-UT). LAMA-UT operates without any language-specific modules while matching the performance of state-of-the-art models trained on a minimal amount of data. Our pipeline consists of two key steps. First, we utilize a universal transcription generator to unify orthographic features into Romanized form and capture common phonetic characteristics across diverse languages. Second, we utilize a universal converter to transform these universal transcriptions into language-specific ones. In experiments, we demonstrate the effectiveness of our proposed method leveraging universal transcriptions for massively multilingual ASR. Our pipeline achieves a relative error reduction rate of 45% when compared to Whisper and performs comparably to MMS, despite being trained on only 0.1% of Whisper's training data. Furthermore, our pipeline does not rely on any language-specific modules. However, it performs on par with zero-shot ASR approaches which utilize additional language-specific lexicons and language models. We expect this framework to serve as a cornerstone for flexible multilingual ASR systems that are generalizable even to unseen languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。