提出新方法让语音识别模型更好区分重叠说话人。
Disentangling Speakers in Multi-Talker Speech Recognition with Speaker-Aware CTC
- 用改进的CTC损失函数显式建模不同说话人在不同时段的语音特征。
- 在低重叠场景下词错误率降低15%，整体降低10%。
- 适合做多说话人语音识别的研究者和开发者参考。
多说话人语音识别（MTASR）面临重叠语音分离与转录的挑战。本文研究将连接时序分类（CTC）结合序列输出训练（SOT）用于MTASR时，在说话人分离中的作用。可视化显示，标准CTC引导编码器在声学嵌入的时间维度上区分不同说话人。基于此发现，我们提出一种新的说话人感知CTC（SACTC）训练目标，基于贝叶斯风险框架设计，通过约束编码器在特定时间帧表示不同说话人的语音单元，实现显式说话人解耦。SOT-SACTC模型在多种重叠程度下均优于标准SOT-CTC，相对词错误率下降10%（整体），低重叠情况下达15%。该工作为基于CTC的多说话人识别提供了新思路。代码已开源：https://github.com/kjw11/Speaker-Aware-CTC。
原文摘要 · Abstract (English)
Multi-talker speech recognition (MTASR) faces unique challenges in disentangling and transcribing overlapping speech. To address these challenges, this paper investigates the role of Connectionist Temporal Classification (CTC) in speaker disentanglement when incorporated with Serialized Output Training (SOT) for MTASR. Our visualization reveals that CTC guides the encoder to represent different speakers in distinct temporal regions of acoustic embeddings. Leveraging this insight, we propose a novel Speaker-Aware CTC (SACTC) training objective, based on the Bayes risk CTC framework. SACTC is a tailored CTC variant for multi-talker scenarios, it explicitly models speaker disentanglement by constraining the encoder to represent different speakers' tokens at specific time frames. When integrated with SOT, the SOT-SACTC model consistently outperforms standard SOT-CTC across various degrees of speech overlap. Specifically, we observe relative word error rate reductions of 10% overall and 15% on low-overlap speech. This work represents an initial exploration of CTC-based enhancements for MTASR tasks, offering a new perspective on speaker disentanglement in multi-talker speech recognition. The code is available at https://github.com/kjw11/Speaker-Aware-CTC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。