针对孟加拉语长音频转录与说话人分离，提出高效适配方案。
An Investigation Into Various Approaches For Bengali Long-Form Speech Transcription and Bengali Speaker Diarization
- 用微调的Whisper模型处理转录，结合自训练分割模型提升精度。
- 在私有榜单上实现0.27的错误率（DER），私有转录误差率38%。
- 成果可推广至南亚低资源语言，代码已开源供复现。
孟加拉语在语音技术中仍属低资源语言，尤其在长音频转录与说话人分离等复杂任务上面临挑战。本文针对Kaggle举办的‘DL Sprint 4.0 - 孟加拉语长音频语音识别’和‘DL Sprint 4.0 - 孟加拉语说话人分离’竞赛，提出多阶段解决方案，旨在解决小时级录音中‘谁在何时说话’的问题。采用在孟加拉语数据上微调的Whisper Medium模型（bengaliAI/tugstugi bengaliai-asr whisper-medium）进行转录，并集成pyannote/speaker-diarization-community-1与自训练的分割模型以应对多样且嘈杂的声学环境。通过两阶段方法结合超参数调优，在私有榜单上实现0.27的错误率（DER），公榜得分为0.19。转录方面，经分块、背景噪声清理及算法后处理，私有榜单词错误率（WER）为0.38。结果表明，针对性调优与策略性数据使用能显著提升南亚语言的AI包容性。相关代码已公开于：https://github.com/Short-Potatoes/Bengali-long-form-transcription-and-diarization.git。
原文摘要 · Abstract (English)
Bengali remains a low-resource language in speech technology, especially for complex tasks like long-form transcription and speaker diarization. This paper presents a multistage approach developed for the "DL Sprint 4.0 - Bengali Long-Form Speech Recognition" and "DL Sprint 4.0 - Bengali Speaker Diarization" competitions on Kaggle, addressing the challenge of "who spoke when/what" in hour-long recordings. We implemented Whisper Medium fine-tuned on Bengali data (bengaliAI/tugstugi bengaliai-asr whisper-medium) for transcription and integrated pyannote/speaker-diarization-community-1 with our custom-trained segmentation model to handle diverse and noisy acoustic environments. Using a two-pass method with hyperparameter tuning, we achieved a DER of 0.27 on the private leaderboard and 0.19 on the public leaderboard. For transcription, chunking, background noise cleaning, and algorithmic post-processing yielded a WER of 0.38 on the private leaderboard. These results show that targeted tuning and strategic data utilization can significantly improve AI inclusivity for South Asian languages. All relevant code is available at: https://github.com/Short-Potatoes/Bengali-long-form-transcription-and-diarization.git Index Terms: Bengali speech recognition, speaker diarization, Whisper, ASR, low-resource languages, pyannote, voice activity detection
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。