arXiv:2411.10879cs.CLcs.AI2024-11中稿 · 2024 IEEE Internat…被引 7

将孟加拉方言语音转为标准孟加拉语,提升跨区域沟通效率。

BanglaDialecto: An End-to-End AI-Powered Regional Speech Standardization

  • 构建端到端系统,将诺阿哈利方言语音转为标准孟加拉语文本
  • 语音识别字错误率仅0.8%,文本翻译BLEU达41.6%
  • 针对方言多样性设计数据集,适合资源匮乏语言研究

本研究聚焦于识别孟加拉国方言,并将多样化的孟加拉语口音转换为标准化的正式孟加拉语语音。方言是特定地区特有的语言变体,表现为发音、语调和词汇差异,受地理、教育水平和社会经济状况影响。方言标准化有助于实现有效沟通、教育一致性、技术可及性、经济机会以及语言资源保护,同时尊重文化多样性。作为全球第五大使用语言,孟加拉语有约55种方言,使用者达1.6亿人,其标准化对开发包容性通信工具至关重要。然而,受限于缺乏全面数据集及处理多样化方言的挑战,相关研究较少。随着多语言大模型(mLLMs)的发展,方言自动语音识别(ASR)与机器翻译(MT)的新可能得以实现。本文提出一个端到端流程,将诺阿哈利方言语音转化为标准孟加拉语语音。研究构建了大规模多样化方言语音数据集,用于优化ASR与大语言模型的微调,实现方言语音转文本和方言文本转标准文本。实验表明,微调Whisper ASR模型达到字错误率(CER)0.8%、词错误率(WER)1.5%;BanglaT5模型在方言到标准文本翻译任务中获得41.6%的BLEU分数。

原文摘要 · Abstract (English)

This study focuses on recognizing Bangladeshi dialects and converting diverse Bengali accents into standardized formal Bengali speech. Dialects, often referred to as regional languages, are distinctive variations of a language spoken in a particular location and are identified by their phonetics, pronunciations, and lexicon. Subtle changes in pronunciation and intonation are also influenced by geographic location, educational attainment, and socioeconomic status. Dialect standardization is needed to ensure effective communication, educational consistency, access to technology, economic opportunities, and the preservation of linguistic resources while respecting cultural diversity. Being the fifth most spoken language with around 55 distinct dialects spoken by 160 million people, addressing Bangla dialects is crucial for developing inclusive communication tools. However, limited research exists due to a lack of comprehensive datasets and the challenges of handling diverse dialects. With the advancement in multilingual Large Language Models (mLLMs), emerging possibilities have been created to address the challenges of dialectal Automated Speech Recognition (ASR) and Machine Translation (MT). This study presents an end-to-end pipeline for converting dialectal Noakhali speech to standard Bangla speech. This investigation includes constructing a large-scale diverse dataset with dialectal speech signals that tailored the fine-tuning process in ASR and LLM for transcribing the dialect speech to dialect text and translating the dialect text to standard Bangla text. Our experiments demonstrated that fine-tuning the Whisper ASR model achieved a CER of 0.8% and WER of 1.5%, while the BanglaT5 model attained a BLEU score of 41.6% for dialect-to-standard text translation.

语音识别方言转换大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。