通过聚类生成说话人标记，提升复杂场景下的多说话人语音识别准确率。
Hypothesis Clustering and Merging: Novel MultiTalker Speech Recognition with Speaker Tokens
- 用聚类生成说话人标记，结合注意力模型实现多说话人分离。
- 在3人混合场景下，干净数据误差降低55%，嘈杂数据降低36%。
- 适合会议、访谈等多人重叠语音的识别任务。
在会议等实际场景中，存在多个未知数量的说话人，且话语常发生重叠。本文提出一种基于注意力机制的编码器-解码器方法，引入通过说话人聚类获得的特殊说话人类别标记。推理时，根据预测的说话人聚类标记选择多个识别假设，并利用归一化编辑距离进行层次聚类合并（AHC），自动确定合适说话人数。在LibriMix数据集上的实验表明，该方法在复杂的3人混合环境下表现优异，相较于传统串行输出训练，在干净数据上相对误差降低55%，在噪声数据上降低36%。
原文摘要 · Abstract (English)
In many real-world scenarios, such as meetings, multiple speakers are present with an unknown number of participants, and their utterances often overlap. We address these multi-speaker challenges by a novel attention-based encoder-decoder method augmented with special speaker class tokens obtained by speaker clustering. During inference, we select multiple recognition hypotheses conditioned on predicted speaker cluster tokens, and these hypotheses are merged by agglomerative hierarchical clustering (AHC) based on the normalized edit distance. The clustered hypotheses result in the multi-speaker transcriptions with the appropriate number of speakers determined by AHC. Our experiments on the LibriMix dataset demonstrate that our proposed method was particularly effective in complex 3-mix environments, achieving a 55% relative error reduction on clean data and a 36% relative error reduction on noisy data compared with conventional serialized output training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。