按语言家族共享连接器，提升多语种语音识别效率与泛化能力
Language Family Matters: Evaluating LLM-Based ASR Across Linguistic Boundaries
- 基于语言家族共享轻量连接器，减少参数量
- 在两个真实语料库上验证，跨领域泛化性能提升
- 适合需要高效部署多语种语音系统的团队
大型语言模型（LLM）驱动的自动语音识别（ASR）系统通过将冻结的语音编码器与预训练的LLM通过轻量连接器相连，在资源有限情况下仍表现优异。以往工作为每种语言单独训练连接器，忽视了语言间的亲缘关系。本文提出一种基于语言家族成员关系的新型连接器共享策略，实现每类语言家族仅需一个连接器。我们在两种多语种LLM和两个真实世界语料库（涵盖精心筛选与众包语音）上实证验证了该方法的有效性。结果表明，基于语言家族的连接器设计在降低参数量的同时，提升了跨领域泛化能力，为多语种ASR部署提供了一种实用且可扩展的方案。
原文摘要 · Abstract (English)
Large Language Model (LLM)-powered Automatic Speech Recognition (ASR) systems achieve strong performance with limited resources by linking a frozen speech encoder to a pretrained LLM via a lightweight connector. Prior work trains a separate connector per language, overlooking linguistic relatedness. We propose an efficient and novel connector-sharing strategy based on linguistic family membership, enabling one connector per family, and empirically validate its effectiveness across two multilingual LLMs and two real-world corpora spanning curated and crowd-sourced speech. Our results show that family-based connectors reduce parameter count while improving generalization across domains, offering a practical and scalable strategy for multilingual ASR deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。