为中亚小语种打造高效语音识别基础模型
GigaAM Multilingual: Foundation Model for Underrepresented Languages
- 用200万小时音频预训练，结合聚类均衡策略
- 在自发口语上性能超越Whisper等模型
- 适合资源匮乏语言的语音系统研究者
尽管近期语音识别模型规模持续扩大，多语言语音识别性能仍不均衡，长尾语言因数据稀缺而表现不佳。本文针对中亚地区代表性不足的语言（哈萨克语、吉尔吉斯语、乌兹别克语），提出GigaAM Multilingual，一个基于Conformer编码器、在200万小时音频上使用HuBERT式目标函数预训练的模型。关键创新在于预训练阶段引入聚类级数据平衡策略，微调阶段采用领域感知采样方法，以缓解主导语言的干扰。在可控对比实验中，该方法在目标语言上的表现优于Whisper Large v3和Omnilingual-1B等强开源预训练编码器，在自发口语任务上取得显著提升，同时保持高效性。论文公开发布基础编码器与语音识别模型，提供一套可在真实数据不平衡条件下有效进行多语言适配的可复现方案。
原文摘要 · Abstract (English)
Despite recent scaling successes, multilingual ASR performance remains highly uneven, with long-tail languages suffering from severe data scarcity. This work addresses the challenge of building robust foundation models for underrepresented Central Asian languages (Kazakh, Kyrgyz, Uzbek). We present GigaAM Multilingual, a Conformer encoder pre-trained on 2M hours of audio using a HuBERT-style objective. Crucially, we introduce a cluster-level data balancing strategy during pre-training and a domain-aware sampling method during fine-tuning to mitigate head-language dominance. In controlled comparisons, our approach outperforms strong open pretrained encoders (Whisper Large v3, Omnilingual-1B) on target languages, achieving significant gains on spontaneous speech while maintaining efficiency. We release the foundation encoder and ASR model, offering a proven recipe for effective multilingual adaptation under realistic data imbalance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。