融合语音与文本低层特征，用混合网络提升情感识别准确率
EmoTech: A Multi-modal Speech Emotion Recognition Using Multi-source Low-level Information with Hybrid Recurrent Network
- 双分支结构分别处理语音和文本的低层特征
- 在相同数据集上达到84%的识别准确率，优于以往方法
- 适合需要多模态情感理解的交互系统研发
情感识别是人机交互中的关键任务，有助于实现更自然、响应更灵敏的系统。本文提出一种多模态情感识别系统，结合语音和文本的低层信息，采用卷积神经网络（CNN）与双向长短期记忆网络（BiLSTM）的混合架构。系统包含两个并行模块：音频模块对梅尔频率倒谱系数（MFCCs）进行处理，分别通过BiLSTM和二维卷积网络提取语音的内在与外在特征；文本模块则利用结合了BiLSTM-CNN的结构，从对应语音的词嵌入中捕捉文本的序列特征。上述低层特征被拼接后，经全连接层分类，实现情感识别。实验表明，该方法在相同数据集上整体准确率达84%，优于已有同类研究。
原文摘要 · Abstract (English)
Emotion recognition is a critical task in human-computer interaction, enabling more intuitive and responsive systems. This study presents a multimodal emotion recognition system that combines low-level information from audio and text, leveraging both Convolutional Neural Networks (CNNs) and Bidirectional Long Short-Term Memory Networks (BiLSTMs). The proposed system consists of two parallel networks: an Audio Block and a Text Block. Mel Frequency Cepstral Coefficients (MFCCs) are extracted and processed by a BiLSTM network and a 2D convolutional network to capture low-level intrinsic and extrinsic features from speech. Simultaneously, a combined BiLSTM-CNN network extracts the low-level sequential nature of text from word embeddings corresponding to the available audio. This low-level information from speech and text is then concatenated and processed by several fully connected layers to classify the speech emotion. Experimental results demonstrate that the proposed EmoTech accurately recognizes emotions from combined audio and text inputs, achieving an overall accuracy of 84%. This solution outperforms previously proposed approaches for the same dataset and modalities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。