用新方法让语音识别模型自动区分多人对话中的说话人。
Speaker-Distinguishable CTC: Learning Speaker Distinction Using CTC for Multi-Talker Speech Recognition
- 在每个音频帧上同时预测语音内容和说话人标签。
- 相比传统方法，错误率降低26%，接近依赖额外信息的顶尖水平。
- 无需时间戳等辅助数据，适合真实对话场景的多说话人识别。
本文提出一种无需额外信息的多说话人语音识别新框架。传统序列输出训练（SOT）因说话人分配失败导致识别错误。尽管引入如标记级时间戳等辅助信息可提升精度，但在自然对话中获取此类信息仍具挑战。为此，我们提出说话人可区分的CTC（SD-CTC），扩展了标准CTC，在每帧上联合预测语音标记及其对应说话人标签。将SD-CTC融入SOT框架后，模型仅通过重叠语音与转录文本即可学习说话人区分能力。实验表明，采用多任务学习的SD-CTC与SOT使错误率降低26%，性能达到依赖辅助信息的最先进方法水平。
原文摘要 · Abstract (English)
This paper presents a novel framework for multi-talker automatic speech recognition without the need for auxiliary information. Serialized Output Training (SOT), a widely used approach, suffers from recognition errors due to speaker assignment failures. Although incorporating auxiliary information, such as token-level timestamps, can improve recognition accuracy, extracting such information from natural conversational speech remains challenging. To address this limitation, we propose Speaker-Distinguishable CTC (SD-CTC), an extension of CTC that jointly assigns a token and its corresponding speaker label to each frame. We further integrate SD-CTC into the SOT framework, enabling the SOT model to learn speaker distinction using only overlapping speech and transcriptions. Experimental comparisons show that multi-task learning with SD-CTC and SOT reduces the error rate of the SOT model by 26% and achieves performance comparable to state-of-the-art methods relying on auxiliary information.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。