用MFCC和CNN-LSTM识别语音情绪,准确率达61.07%。
Speech Emotion Detection Based on MFCC and CNN-LSTM Architecture
- 结合MFCC特征与混合CNN-LSTM模型处理语音序列数据。
- 整体测试准确率61.07%,愤怒和中性情绪识别最高达75.31%和71.70%。
- 上下文依赖情绪(如惊讶)易混淆,负面情绪间也常误判。
情感检测技术广泛应用于面部图像与语音音频特征分析,其中语音处理因复杂性及特征提取困难仍具挑战。本文选用SAVEE与RAVDESS数据集的部分样本,构建包含七类常见情绪(快乐、中性、悲伤、愤怒、厌恶、恐惧、惊讶)的语料库,共数千样本。基于Librosa工具包,将原始音频转化为波形图与频谱图进行分析,并聚焦于多种特征,特别是梅尔频率倒谱系数(MFCC)。采用具备时序建模能力的混合CNN-LSTM架构,由四层卷积层与三层长短期记忆层组成。实验结果表明,该模型在测试集上综合准确率达61.07%,其中愤怒与中性情绪识别分别达到75.31%和71.70%。分类性能受情绪类型影响显著:高频且特征明显的类别误判率较低;而依赖上下文的惊讶情绪更易被误判为正负情绪,负面情绪之间亦存在混淆现象。
原文摘要 · Abstract (English)
Emotion detection techniques have been applied to multiple cases mainly from facial image features and vocal audio features, of which the latter aspect is disputed yet not only due to the complexity of speech audio processing but also the difficulties of extracting appropriate features. Part of the SAVEE and RAVDESS datasets are selected and combined as the dataset, containing seven sorts of common emotions (i.e. happy, neutral, sad, anger, disgust, fear, and surprise) and thousands of samples. Based on the Librosa package, this paper processes the initial audio input into waveplot and spectrum for analysis and concentrates on multiple features including MFCC as targets for feature extraction. The hybrid CNN-LSTM architecture is adopted by virtue of its strong capability to deal with sequential data and time series, which mainly consists of four convolutional layers and three long short-term memory layers. As a result, the architecture achieved an accuracy of 61.07% comprehensively for the test set, among which the detection of anger and neutral reaches a performance of 75.31% and 71.70% respectively. It can also be concluded that the classification accuracy is dependent on the properties of emotion to some extent, with frequently-used and distinct-featured emotions having less probability to be misclassified into other categories. Emotions like surprise whose meaning depends on the specific context are more likely to confuse with positive or negative emotions, and negative emotions also have a possibility to get mixed with each other.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。