用深度学习帮阿拉伯语儿童纠正发音，准确率达99%
A Novel Speech Analysis and Correction Tool for Arabic-Speaking Children
- 将语音转图像，用ResNet18识别发音错误
- 99.015%准确率，梅尔频谱图优于梅尔倒谱系数
- 游戏化界面激励孩子练习，适合语言康复场景
本文提出名为ArPA的新应用，用于帮助阿拉伯语儿童改善发音问题。系统包含诊断与治疗两个模块：诊断阶段通过采集语音信号，经预处理后使用KNN、SVM、决策树及ResNet18等机器学习与深度神经网络分类器进行分析；治疗模块采用趣味化游戏界面，正确发音可提升角色等级，实现正向激励。实验使用两个数据集：一家托儿中心采集的数据与阿拉伯字母发音录音集。创新性地采用梅尔频谱图（Mel-Spectrogram）和梅尔倒谱系数（MFCC）图像作为输入特征，结果表明，基于梅尔频谱图的ResNet18在识别阿拉伯语发音错误上达到99.015%的准确率，显著优于使用MFCC图像的版本。
原文摘要 · Abstract (English)
This paper introduces a new application named ArPA for Arabic kids who have trouble with pronunciation. Our application comprises two key components: the diagnostic module and the therapeutic module. The diagnostic process involves capturing the child's speech signal, preprocessing, and analyzing it using different machine learning classifiers like K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Decision Trees as well as deep neural network classifiers like ResNet18. The therapeutic module offers eye-catching gamified interfaces in which each correctly spoken letter earns a higher avatar level, providing positive reinforcement for the child's pronunciation improvement. Two datasets were used for experimental evaluation: one from a childcare centre and the other including Arabic alphabet pronunciation recordings. Our work uses a novel technique for speech recognition using Melspectrogram and MFCC images. The results show that the ResNet18 classifier on speech-to-image converted data effectively identifies mispronunciations in Arabic speech with an accuracy of 99.015\% with Mel-Spectrogram images outperforming ResNet18 with MFCC images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。