用弱标注数据训练低资源语言语音翻译模型，效果媲美大规模基线。
End-to-End Speech Translation for Low-Resource Languages Using Weakly Labeled Data
- 基于句向量挖掘多语种语料，构建弱标注语音翻译数据集。
- 在4个印地语方言对上，弱标注数据训练模型性能接近主流基线。
- 适合低资源语言语音翻译研究者参考，尤其关注数据效率的场景。
高质量标注数据的匮乏是构建高效端到端语音转文本翻译（ST）系统的主要挑战，尤其是在低资源语言中。本文提出假设：弱标注数据可用于构建低资源语言对的ST模型。我们借助先进的句子编码器进行双语语料挖掘，构建了包含孟加拉语-印地语、马拉雅拉姆语-印地语、奥迪语-印地语和泰卢固语-印地语四种语言对的语音转文本翻译数据集Shrutilipi-anuvaad。通过构建不同质量与数量的训练数据版本，探究了弱标注数据的质量与数量对模型性能的影响。结果表明，仅使用弱标注数据即可构建有效的ST系统，其性能可与大规模多模态多语言基线模型如SONAR和SeamlessM4T相媲美。
原文摘要 · Abstract (English)
The scarcity of high-quality annotated data presents a significant challenge in developing effective end-to-end speech-to-text translation (ST) systems, particularly for low-resource languages. This paper explores the hypothesis that weakly labeled data can be used to build ST models for low-resource language pairs. We constructed speech-to-text translation datasets with the help of bitext mining using state-of-the-art sentence encoders. We mined the multilingual Shrutilipi corpus to build Shrutilipi-anuvaad, a dataset comprising ST data for language pairs Bengali-Hindi, Malayalam-Hindi, Odia-Hindi, and Telugu-Hindi. We created multiple versions of training data with varying degrees of quality and quantity to investigate the effect of quality versus quantity of weakly labeled data on ST model performance. Results demonstrate that ST systems can be built using weakly labeled data, with performance comparable to massive multi-modal multilingual baselines such as SONAR and SeamlessM4T.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。