用乱序模型实现多人重叠语音的一次性对齐与说话人识别
Modeling Overlapped Speech with Shuffles
- 基于乱序积和部分有序自动机建模重叠语音流
- 在合成LibriSpeech数据上实现端到端的单次对齐，准确率超90%
- 适合语音识别、多说话人对话系统研究者使用
本文提出利用乱序积和部分有序有限状态自动机（FSAs）来建模并行数据流，如重叠语音。通过在子词、词和短语级别对所有可能的序列化进行边际化，以总得分作为损失函数进行训练。为减小图规模，引入时间约束构建部分有序FSAs。通过直接建模（标记, 说话人）元组实现说话人归属。经由乱序积自动机的维特比对齐可实现一次遍历完成对齐。在合成LibriSpeech重叠语音数据上评估性能。据我们所知，这是首个实现多说话人录音单次对齐的算法。所有算法均基于k2 / Icefall实现。
原文摘要 · Abstract (English)
We propose to model parallel streams of data, such as overlapped speech, using shuffles. Specifically, this paper shows how the shuffle product and partial order finite-state automata (FSAs) can be used for alignment and speaker-attributed transcription of overlapped speech. We train using the total score on these FSAs as a loss function, marginalizing over all possible serializations of overlapping sequences at subword, word, and phrase levels. To reduce graph size, we impose temporal constraints by constructing partial order FSAs. We address speaker attribution by modeling (token, speaker) tuples directly. Viterbi alignment through the shuffle product FSA directly enables one-pass alignment. We evaluate performance on synthetic LibriSpeech overlaps. To our knowledge, this is the first algorithm that enables single-pass alignment of multi-talker recordings. All algorithms are implemented using k2 / Icefall.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。