评测5大主流ASR对非母语英语的识别效果,发现Whisper在朗读中表现最佳。
Automatic Speech Recognition for Non-Native English: Accuracy and Disfluency Handling
- 对比五种主流ASR系统在非母语英语上的表现,使用六种语言背景的语音数据。
- 朗读语音下Whisper和AssemblyAI的错误率最低(平均MER 0.054/0.056),接近人类水平。
- 不同系统对填充词、重复等不流畅表达处理差异大,适合特定场景的选型参考。
自动语音识别(ASR)在计算机辅助语言学习(CALL)和测试(CALT)中已应用多年。本研究评估了五种前沿ASR系统在非母语英语语音上的识别准确率,使用来自六种不同母语背景(阿拉伯语、中文、印地语、韩语、西班牙语、越南语)的L2-ARCTIC语料库数据,涵盖朗读与即兴表达两种形式。朗读部分包含24名发音人共2400句单句录音,即兴表达部分包含22名发言人的叙事录音。结果显示,在朗读语音中,Whisper和AssemblyAI表现最佳,平均匹配错误率(MER)分别为0.054和0.056,接近人类水平;在即兴表达中,RevAI表现最优,平均MER为0.063。研究还分析了各系统对填充词、重复、修正等不流畅表达的处理能力,发现系统间表现差异显著。尽管处理速度差异明显,但更慢并不意味着更高准确率。本研究旨在帮助语言教学者与研究者了解各系统的优劣,选择适配特定应用场景的ASR工具。
原文摘要 · Abstract (English)
Automatic speech recognition (ASR) has been an essential component of computer assisted language learning (CALL) and computer assisted language testing (CALT) for many years. As this technology continues to develop rapidly, it is important to evaluate the accuracy of current ASR systems for language learning applications. This study assesses five cutting-edge ASR systems' recognition of non-native accented English speech using recordings from the L2-ARCTIC corpus, featuring speakers from six different L1 backgrounds (Arabic, Chinese, Hindi, Korean, Spanish, and Vietnamese), in the form of both read and spontaneous speech. The read speech consisted of 2,400 single sentence recordings from 24 speakers, while the spontaneous speech included narrative recordings from 22 speakers. Results showed that for read speech, Whisper and AssemblyAI achieved the best accuracy with mean Match Error Rates (MER) of 0.054 and 0.056 respectively, approaching human-level accuracy. For spontaneous speech, RevAI performed best with a mean MER of 0.063. The study also examined how each system handled disfluencies such as filler words, repetitions, and revisions, finding significant variation in performance across systems and disfluency types. While processing speed varied considerably between systems, longer processing times did not necessarily correlate with better accuracy. By detailing the performance of several of the most recent, widely-available ASR systems on non-native English speech, this study aims to help language instructors and researchers understand the strengths and weaknesses of each system and identify which may be suitable for specific use cases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。