首个面向孟加拉方言的实时语音助手，解决低资源语言的可用性问题。
BanglaTalk: Towards Real-Time Speech Assistance for Bengali Regional Dialects
- 采用客户端-服务器架构与RTP协议，实现低延迟通信。
- 自研方言感知模型BRDialect，在10种方言上提升12.41%-33.98%识别率。
- 仅需24kbps带宽，端到端延迟仅4.9秒，适合低资源场景使用。
实时语音助手正日益普及，以提升信息可及性。孟加拉语作为低资源语言，存在高度区域方言多样性，现有系统未针对实时应用优化，且仅支持标准孟加拉语。本文提出首个面向孟加拉方言的实时语音辅助系统BanglaTalk，采用客户端-服务器架构，利用实时传输协议（RTP）保障低延迟通信。为应对方言差异，我们基于IndicWav2Vec模型在10种孟加拉方言上微调，构建方言感知自动语音识别系统BRDialect，其在RegSpeech12数据集上相较基线模型性能提升12.41%-33.98%。此外，BanglaTalk可在24 kbps低带宽下运行，平均端到端延迟仅为4.9秒，兼具成本效益与交互性，为多样化的孟加拉语使用者提供包容性语音技术。代码已开源：https://github.com/Jak57/BanglaTalk
原文摘要 · Abstract (English)
Real-time speech assistants are becoming increasingly popular for ensuring improved accessibility to information. Bengali, being a low-resource language with a high regional dialectal diversity, has seen limited progress in developing such systems. Existing systems are not optimized for real-time use and focus only on standard Bengali. In this work, we present BanglaTalk, the first real-time speech assistance system for Bengali regional dialects. BanglaTalk follows the client-server architecture and uses the Real-time Transport Protocol (RTP) to ensure low-latency communication. To address dialectal variation, we introduce a dialect-aware ASR system, BRDialect, developed by fine-tuning the IndicWav2Vec model in ten Bengali regional dialects. It outperforms the baseline ASR models by 12.41-33.98% on the RegSpeech12 dataset. Furthermore, BanglaTalk can operate at a low bandwidth of 24 kbps while maintaining an average end-to-end delay of 4.9 seconds. Low bandwidth usage and minimal end-to-end delay make the system both cost-effective and interactive for real-time use cases, enabling inclusive and accessible speech technology for the diverse community of Bengali speakers. Code is available in https://github.com/Jak57/BanglaTalk
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。