用外部信息提升阿拉伯语零样本语音识别,不改模型也能降错率。
Zero-Shot Context-Aware ASR for Diverse Arabic Varieties
- 通过提示词和检索样例,在推理时引入上下文信息。
- 多语种阿拉伯语识别错误率平均降低22.29%(标准语)至9.15%(方言)。
- 适用于无须训练的场景,尤其适合口语和多人对话识别。
阿拉伯语零样本语音识别仍具挑战:尽管多语言模型在现代标准阿拉伯语(MSA)上表现良好,但方言和带口音的语音错误率显著上升,源于语言差异和标注数据稀缺。本文研究上下文感知解码作为一种轻量级测试时适应范式,在不更新参数的前提下,利用外部辅助信息进行推理。针对可提示的编码器-解码器语音识别模型(如Whisper),采用(i)基于首轮识别结果的解码器提示,及(ii)通过检索语音-文本样例进行编码器/解码器前缀添加,并结合简单提示重排与可选的说话人匹配合成样例,以增强非正式及多说话人场景下的鲁棒性。为拓展该方法至非可提示架构,提出代理引导的n-best选择:给定一个或多个外部代理假设,从模型输出的n-best候选中选取与代理文本距离最小者,实现无需直接提示的上下文推理。在涵盖10种阿拉伯语条件(包括MSA、带口音的MSA及多种方言)的实验中,上下文感知解码在MSA上实现22.29%的相对词错误率(WER)降低,在带口音的MSA上降低20.54%,在方言上降低9.15%。对于CTC模型,代理引导选择在MSA上实现15.6%的相对WER降低,并恢复了大量原始n-best选择的性能增益,证明该方法可泛化至非编码器-解码器架构。
原文摘要 · Abstract (English)
Zero-shot ASR for Arabic remains challenging: while multilingual models perform well on Modern Standard Arabic (MSA), error rates rise sharply on dialectal and accented speech due to linguistic mismatch and scarce labeled data. We study context-aware decoding as a lightweight test-time adaptation paradigm that conditions inference on external side information without parameter updates. For promptable encoder-decoder ASR (e.g., Whisper), we incorporate context through (i) decoder prompting with first-pass hypotheses and (ii) encoder/decoder prefixing with retrieved speech-text exemplars, complemented by simple prompt reordering and optional speaker-matched synthetic exemplars to improve robustness in informal and multi-speaker settings. To extend contextual adaptation beyond promptable architectures, we introduce proxy-guided n-best selection for CTC ASR: given one or more external proxy hypotheses, we select from a model's n-best list by minimizing text-level distance to the proxies, enabling contextual inference without direct prompting. Across ten Arabic conditions spanning MSA, accented MSA, and multiple dialects, context-aware decoding yields average relative WER reductions of 22.29% on MSA, 20.54 on accented MSA, and 9.15% on dialectal Arabic. For CTC models, proxy-guided selection reduces WER by 15.6% relative on MSA and recovers a substantial fraction of oracle n-best gains, demonstrating that context-aware inference generalizes beyond encoder-decoder ASR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。