让语音识别模型自动检测口语中的停顿和重复，无需重新训练。
Augmenting Automatic Speech Recognition Models with Disfluency Detection
- 用改进的CTC对齐算法捕捉语音中每个词的时间戳。
- 通过分析时间戳间隙，识别出74.13%原本被漏掉的词语。
- 不需微调模型，可适用于任意现有语音识别系统。
口语中常见话语不流畅现象，但标准语音识别（ASR）模型因训练数据多为流畅文本，难以准确识别。现有研究多关注转录文本中的不流畅检测，忽视其在语音中的具体位置与持续时间。以往方法常需模型微调，且仅覆盖有限类型的不流畅。本文提出一种仅依赖推理的方案，可为任意ASR模型增加开放集不流畅检测能力。首先验证了ASR模型在识别不流畅语音上的困难；随后提出基于连接时序分类（CTC）的改进强制对齐算法，以预测词级时间戳并有效捕捉不流畅语音；同时设计一个分类模型，将时间戳间隙判别为不流畅语音或静音，准确率达81.62%，F1得分为80.07%。在不流畅语音数据集上测试该增强流程，成功捕获了74.13%原转录中遗漏的词语，展现了其在下游任务中的潜力。
原文摘要 · Abstract (English)
Speech disfluency commonly occurs in conversational and spontaneous speech. However, standard Automatic Speech Recognition (ASR) models struggle to accurately recognize these disfluencies because they are typically trained on fluent transcripts. Current research mainly focuses on detecting disfluencies within transcripts, overlooking their exact location and duration in the speech. Additionally, previous work often requires model fine-tuning and addresses limited types of disfluencies. In this work, we present an inference-only approach to augment any ASR model with the ability to detect open-set disfluencies. We first demonstrate that ASR models have difficulty transcribing speech disfluencies. Next, this work proposes a modified Connectionist Temporal Classification(CTC)-based forced alignment algorithm from \cite{kurzinger2020ctc} to predict word-level timestamps while effectively capturing disfluent speech. Additionally, we develop a model to classify alignment gaps between timestamps as either containing disfluent speech or silence. This model achieves an accuracy of 81.62% and an F1-score of 80.07%. We test the augmentation pipeline of alignment gap detection and classification on a disfluent dataset. Our results show that we captured 74.13% of the words that were initially missed by the transcription, demonstrating the potential of this pipeline for downstream tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。