针对孟加拉语长音频，用时间戳分段+定制化微调，提升语音识别与说话人分离准确率。
WhisperAlign: Word-Boundary-Aware ASR and WhisperX-Anchored Pyannote Diarization for Long-Form Bengali Speech
- 用Whisper时间戳智能切分长音频，保证上下文连续性。
- 在竞赛数据集上微调Pyannote模型，降低复杂重叠对话的错误率。
- 适合低资源语言语音处理，尤其关注多说话人场景。
本文提出DL Sprint 4.0中针对孟加拉语长时序语音识别（任务1）与说话人分离（任务2）的解决方案。长时序、多说话人语音带来语音活动检测、重叠语音和上下文保持等挑战。为应对语音识别难题，我们采用Whisper-timestamped实现鲁棒的音频分块策略，将精确的时间对齐片段输入微调后的声学模型，实现高精度转写。对于说话人分离任务，构建了融合pyannote.audio与WhisperX的集成流水线。关键创新在于在比赛数据集上对Pyannote分割模型进行领域专用微调，使其更精准捕捉孟加拉语对话特征，有效解析复杂重叠的说话人边界。实验表明，结合时间戳分块与针对性分割微调，显著降低低资源条件下的词错误率（WER）与说话人分离错误率（DER）。
原文摘要 · Abstract (English)
This paper presents our solution for the DL Sprint 4.0, addressing the dual challenges of Bengali Long-Form Speech Recognition (Task 1) and Speaker Diarization (Task 2). Processing long-form, multi-speaker Bengali audio introduces significant hurdles in voice activity detection, overlapping speech, and context preservation. To solve the long-form transcription challenge, we implemented a robust audio chunking strategy utilizing whisper-timestamped, allowing us to feed precise, context-aware segments into our fine-tuned acoustic model for high-accuracy transcription. For the diarization task, we developed an integrated pipeline leveraging pyannote.audio and WhisperX. A key contribution of our approach is the domain-specific fine-tuning of the Pyannote segmentation model on the competition dataset. This adaptation allowed the model to better capture the nuances of Bengali conversational dynamics and accurately resolve complex, overlapping speaker boundaries. Our methodology demonstrates that applying intelligent timestamped chunking to ASR and targeted segmentation fine-tuning to diarization significantly drives down Word Error Rate (WER) and Diarization Error Rate (DER), in low-resource settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。