用融合模型提升口吃检测精度,兼顾类型与严重程度判断。
An End-To-End Stuttering Detection Method Based On Conformer And BILSTM
- 结合Conformer与双向LSTM提取声学特征并捕捉上下文
- 多任务学习策略使平均F1提升39.8%(对比基线)
- 适合语音病理评估与临床辅助诊断场景
口吃是一种神经发育性语言障碍,表现为停顿、重复、延长等常见症状。言语语言病理学家通常通过观察这些症状来评估口吃的类型与严重程度。尽管已有多种有效的端到端口吃检测方法,但这一过程中的任务间关系尚不明确,合理运用多任务学习可提升检测性能。本文提出一种新型口吃事件检测模型,旨在帮助言语语言病理学家同时评估口吃类型与严重程度。首先,Conformer模型从口吃语音中提取声学特征,随后通过双向长短期记忆网络(BiLSTM)捕获上下文信息。此外,我们探索了口吃检测的多任务学习策略,并提出一种高效方法。实验结果表明,该模型在基于AS-70数据集的SLT 2024口吃语音挑战赛中,相比基线方法平均F1分数提升24.8%,获得第一名。进一步的消融实验显示,所提方法相较基线平均F1提升39.8%。
原文摘要 · Abstract (English)
Stuttering is a neurodevelopmental speech disorder characterized by common speech symptoms such as pauses, exclamations, repetition, and prolongation. Speech-language pathologists typically assess the type and severity of stuttering by observing these symptoms. Many effective end-to-end methods exist for stuttering detection, but a commonly overlooked challenge is the uncertain relationship between tasks involved in this process. Using a suitable multi-task strategy could improve stuttering detection performance. This paper presents a novel stuttering event detection model designed to help speech-language pathologists assess both the type and severity of stuttering. First, the Conformer model extracts acoustic features from stuttered speech, followed by a Long Short-Term Memory (LSTM) network to capture contextual information. Finally, we explore multi-task learning for stuttering and propose an effective multi-task strategy. Experimental results show that our model outperforms current state-of-the-art methods for stuttering detection. In the SLT 2024 Stuttering Speech Challenge based on the AS-70 dataset [1], our model improved the mean F1 score by 24.8% compared to the baseline method and achieved first place. On this basis, we conducted relevant extensive experiments on LSTM and multi-task learning strategies respectively. The results show that our proposed method improved the mean F1 score by 39.8% compared to the baseline method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。