无需对齐的多说话人语音识别新方法,仅用一次编码器处理即可识别多人对话。
Alignment-Free Training for Transducer-based Multi-Talker ASR
- 通过在转录前添加说话人提示标记,实现无对齐训练
- 在Switchboard数据集上达到9.8%的词错误率,媲美先进方法
- 适合希望简化多说话人识别系统部署的研究者和工程师
将循环神经网络转换器(RNNT)扩展至多说话人语音识别对于更广泛的应用至关重要。传统多说话人RNNT(MT-RNNT)依赖多个编码器或解码器,或需将所有说话人的转录序列串行化,前者计算开销大,后者需依赖外部ASR系统获取精确时间戳。本文提出一种新型无对齐训练方案(MT-RNNT-AFT),采用标准RNNT架构,在每段转录前加入对应说话人的提示标记以反映其出现顺序。该方法无需准确对齐,仅需一次编码器处理即可识别所有说话人语音。实验表明,MT-RNNT-AFT在Switchboard数据集上达到9.8%的词错误率,性能与现有最佳方法相当,同时大幅简化训练流程。
原文摘要 · Abstract (English)
Extending the RNN Transducer (RNNT) to recognize multi-talker speech is essential for wider automatic speech recognition (ASR) applications. Multi-talker RNNT (MT-RNNT) aims to achieve recognition without relying on costly front-end source separation. MT-RNNT is conventionally implemented using architectures with multiple encoders or decoders, or by serializing all speakers' transcriptions into a single output stream. The first approach is computationally expensive, particularly due to the need for multiple encoder processing. In contrast, the second approach involves a complex label generation process, requiring accurate timestamps of all words spoken by all speakers in the mixture, obtained from an external ASR system. In this paper, we propose a novel alignment-free training scheme for the MT-RNNT (MT-RNNT-AFT) that adopts the standard RNNT architecture. The target labels are created by appending a prompt token corresponding to each speaker at the beginning of the transcription, reflecting the order of each speaker's appearance in the mixtures. Thus, MT-RNNT-AFT can be trained without relying on accurate alignments, and it can recognize all speakers' speech with just one round of encoder processing. Experiments show that MT-RNNT-AFT achieves performance comparable to that of the state-of-the-art alternatives, while greatly simplifying the training process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。