构建跨语系低资源语音识别评测基准，揭示大模型在小语种上的表现短板。
LoASR-Bench: Evaluating Large Speech Language Models on Low-Resource Automatic Speech Recognition Across Language Families
- 设计覆盖9大语系25种语言的跨语言跨书写系统评测集
- 实测最新语音大模型在低资源语言上性能显著下降
- 适合关注多语言语音系统落地的研究者与开发者
大型语言模型（LLMs）推动了语音语言模型（SpeechLMs）的发展，在高资源条件下实现了出色的自动语音识别（ASR）性能。然而，现有评测基准主要聚焦高资源语言，导致对SpeechLMs在低资源语言中的行为理解不足。这一差距至关重要，因为实际语音识别系统必须可靠支持低资源语言，并在不同语系间具备泛化能力，否则将直接影响基于SpeechLM的ASR在真实多语言场景中的部署。为此，我们提出 extbf{LoASR-Bench}，一个全面评估最新SpeechLMs在跨语系低资源语音识别中表现的基准。该基准包含来自9个语言家族的25种语言，涵盖拉丁文及非拉丁文字，支持跨语言与跨书写系统的ASR性能评估。实验结果凸显了当前最先进SpeechLMs在处理真实低资源语言时的局限性。
原文摘要 · Abstract (English)
Large language models (LLMs) have driven substantial advances in speech language models (SpeechLMs), yielding strong performance in automatic speech recognition (ASR) under high-resource conditions. However, existing benchmarks predominantly focus on high-resource languages, leaving the ASR behavior of SpeechLMs in low-resource languages insufficiently understood. This gap is critical, as practical ASR systems must reliably support low-resource languages and generalize across diverse language families, and it directly hinders the deployment of SpeechLM-based ASR in real-world multilingual scenarios. As a result, it is essential to evaluate SpeechLMs on low-resource languages to ensure their generalizability across different language families. To address this problem, we propose \textbf{LoASR-Bench}, a comprehensive benchmark designed to evaluate \textbf{lo}w-resource \textbf{a}utomatic \textbf{s}peech \textbf{r}ecognition (\textbf{ASR}) of the latest SpeechLMs across diverse language families. LoASR-Bench comprises 25 languages from 9 language families, featuring both Latin and non-Latin scripts, enabling cross-linguistic and cross-script assessment of ASR performance of current SpeechLMs. Experimental results highlight the limitations of the latest SpeechLMs in handling real-world low-resource languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。