分析两种濒危语言的语音识别,发现数据少才是主要问题。
Hard to Be Heard: Phoneme-Level ASR Analysis of Phonologically Complex, Low-Resource Endangered Languages

- 用自定义音素词表优化wav2vec2模型,提升识别效果
- 音素识别率随训练数据量呈类S型增长,数据少是主因
- 适合研究低资源语言语音识别或濒危语言保护的研究者
我们对两种低资源、音系复杂的东高加索语言Archi和Rutul进行了音素级自动语音识别(ASR)分析,基于约50分钟和1小时20分钟的标准化语音-转录数据。现有录音与转写被整合并处理为适合训练与评估的格式。评估了wav2vec2、Whisper和Qwen2-Audio等前沿音频与音频-语言模型。针对wav2vec2,我们引入基于启发式输出层初始化的语言特定音素词表,在极低资源条件下实现稳定提升,性能可媲美甚至超过Whisper。除标准词错误率和字符错误率外,还进行了详细的音素级错误分析。结果表明,音素识别准确率与训练频率强相关,呈现典型的S型学习曲线;而Archi语言中Whisper的表现部分偏离该规律,暗示模型存在超越训练频率的泛化特性。总体而言,许多看似由音系复杂性导致的错误,实则更应归因于数据稀缺。研究证明音素级评估在理解低资源、类型复杂语言中ASR行为的重要价值。
原文摘要 · Abstract (English)
We present a phoneme-level analysis of automatic speech recognition (ASR) for two low-resourced and phonologically complex East Caucasian languages, Archi and Rutul, based on curated and standardized speech-transcript resources totaling approximately 50 minutes and 1 hour 20 minutes of audio, respectively. Existing recordings and transcriptions are consolidated and processed into a form suitable for ASR training and evaluation. We evaluate several state-of-the-art audio and audio-language models, including wav2vec2, Whisper, and Qwen2-Audio. For wav2vec2, we introduce a language-specific phoneme vocabulary with heuristic output-layer initialization, which yields consistent improvements and achieves performance comparable to or exceeding Whisper in these extremely low-resource settings. Beyond standard word and character error rates, we conduct a detailed phoneme-level error analysis. We find that phoneme recognition accuracy strongly correlates with training frequency, exhibiting a characteristic sigmoid-shaped learning curve. For Archi, this relationship partially breaks for Whisper, pointing to model-specific generalization effects beyond what is predicted by training frequency. Overall, our results indicate that many errors attributed to phonological complexity are better explained by data scarcity. These findings demonstrate the value of phoneme-level evaluation for understanding ASR behavior in low-resource, typologically complex languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。