将英语语音情感保留地翻译成阿拉伯语语音，适用于客服等跨语言场景。
EmoAra: Emotion-Preserving English Speech Transcription and Cross-Lingual Translation with Arabic Text-to-Speech
- 端到端流程整合语音情感识别、转写、翻译与合成，保持情绪特征。
- 情感分类F1达94%，翻译BLEU为56，人类评估满意度81%。
- 适合金融客服等需保留情绪的跨语言语音交互应用。
本文提出EmoAra，一种用于跨语言语音通信的端到端情感保留系统，针对银行客户服务中情绪影响服务品质的场景。该系统集成语音情感识别、自动语音识别、机器翻译和文本转语音技术，将英文语音转化为保留情感色彩的阿拉伯语语音输出。采用基于CNN的情感分类器，Whisper进行英文转写，微调后的MarianMT模型实现英阿翻译，MMS-TTS-Ara生成阿拉伯语语音。实验显示，情感分类F1得分为94%，翻译性能达到BLEU 56和BERTScore F1 88.7%，在银行领域翻译的人类评估平均得分81%。相关代码与资源已开源至配套GitHub仓库。
原文摘要 · Abstract (English)
This work presents EmoAra, an end-to-end emotion-preserving pipeline for cross-lingual spoken communication, motivated by banking customer service where emotional context affects service quality. EmoAra integrates Speech Emotion Recognition, Automatic Speech Recognition, Machine Translation, and Text-to-Speech to process English speech and deliver an Arabic spoken output while retaining emotional nuance. The system uses a CNN-based emotion classifier, Whisper for English transcription, a fine-tuned MarianMT model for English-to-Arabic translation, and MMS-TTS-Ara for Arabic speech synthesis. Experiments report an F1-score of 94% for emotion classification, translation performance of BLEU 56 and BERTScore F1 88.7%, and an average human evaluation score of 81% on banking-domain translations. The implementation and resources are available at the accompanying GitHub repository.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。