让1600多种语言实现低资源语音识别,社区只需少量数据即可接入。
Omnilingual ASR: Open-Source Multilingual Speech Recognition for 1600+ Languages
- 用70亿参数自监督预训练+类大模型解码器,支持零样本泛化。
- 仅需少量数据即可为新语言添加识别能力,覆盖超1600种语言。
- 开源模型支持从300M到70亿参数,助力研究与社区参与。
自动语音识别(ASR)在高资源语言上进展显著,但全球7000多种语言中绝大多数仍无支持,大量长尾语言被忽视。现有技术扩展成本高、架构限制多,且缺乏社区协作易引发伦理问题。为此,我们提出首个可扩展的通用多语言语音识别系统Omnilingual ASR,使社区仅需少量数据样本即可引入未覆盖语言。该系统将自监督预训练规模扩展至70亿参数,构建具备零样本泛化能力的编码器-解码器架构,采用类大模型设计的解码器。训练语料涵盖广泛语言多样性,学习出对未见语言具有鲁棒性的语音表征。结合公开数据与通过有偿本地合作收集的社区录音,系统覆盖超过1600种语言,其中500余种为此前从未被ASR支持的语言。自动评估显示其在低资源条件下显著优于以往系统,并具备强泛化能力。我们发布系列模型,包括300M(低功耗设备适用)和70亿参数版本。我们反思设计中的伦理考量,强调开源可降低研究与社区参与门槛,推动新型协作。相关代码与模型已开源:https://github.com/facebookresearch/omnilingual-asr。
原文摘要 · Abstract (English)
Automatic speech recognition (ASR) has advanced in high-resource languages, but most of the world's 7,000+ languages remain unsupported, leaving thousands of long-tail languages behind. Expanding ASR coverage has been costly and limited by architectures that restrict language support, making extension inaccessible to most--all while entangled with ethical concerns when pursued without community collaboration. To transcend these limitations, we introduce Omnilingual ASR, the first large-scale ASR system designed for extensibility. Omnilingual ASR enables communities to introduce unserved languages with only a handful of data samples. It scales self-supervised pre-training to 7B parameters to learn robust speech representations and introduces an encoder-decoder architecture designed for zero-shot generalization, leveraging a LLM-inspired decoder. This capability is grounded in a massive and diverse training corpus; by combining breadth of coverage with linguistic variety, the model learns representations robust enough to adapt to unseen languages. Incorporating public resources with community-sourced recordings gathered through compensated local partnerships, Omnilingual ASR expands coverage to over 1,600 languages, the largest such effort to date--including over 500 never before served by ASR. Automatic evaluations show substantial gains over prior systems, especially in low-resource conditions, and strong generalization. We release Omnilingual ASR as a family of models, from 300M variants for low-power devices to 7B for maximum accuracy. We reflect on the ethical considerations shaping this design and conclude by discussing its societal impact. In particular, we highlight how open-sourcing models and tools can lower barriers for researchers and communities, inviting new forms of participation. Open-source artifacts are available at https://github.com/facebookresearch/omnilingual-asr.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。