让AI自动生成情绪并实时合成情感语音，更自然地对话。
Self-EmoQ: Plutchik-Guided Value-based Planning to Drive Streaming Emotional TTS
- 用大模型预判情绪，再生成对应情感的语音，实现流式输出。
- 在多个数据集上优于提示和微调方法，情绪判断准确率提升12%以上。
- 适合需要真实情感交互的聊天机器人、语音助手等场景。
情感交互对对话式AI愈发重要，但现有系统缺乏自我情绪判断机制来驱动流式文本到语音（TTS）合成。本文提出一种基于情绪规划的框架，在文本生成前确定情绪，并以流式方式支撑下游情感TTS。该框架通过一个即插即用的LLM模块实现，初始值来自预训练LLM，采用强化学习（RL）进行训练，将情绪作为动作。设计了混合奖励机制，结合模仿信号与基于普拉奇克情绪轮理论的评分体系。在DailyDialog、EmoryNLP、IMEOCAP和MELD数据集上的实验表明，该方法在情绪判断与回复质量方面均优于提示和微调基线。最终构建了完整的流式部署管道，语音质量验证了框架在情绪一致性、上下文连贯性及表达流畅性上的优势。代码、案例与演示见https://sixingdeguo.github.io/EmoQ-page/。
原文摘要 · Abstract (English)
Emotional interaction is increasingly crucial for conversational AI, yet current systems lack a self-emotion determination mechanism to drive the streaming text-to-speech (TTS) synthesis. We propose an emotion-planning framework that determines the emotion prior to the textual generation, grounding the downstream emotional TTS in a streaming manner. The framework is implemented by a plug-and-play LLM module, initialized from pretrained LLMs, and trained by reinforcement learning (RL) with emotions as the actions. A hybrid reward is employed which combines imitation signals with theory-driven scoring, in which the theory of Plutchik's wheel of emotions is adopted. By experiments on DailyDialog, EmoryNLP, IMEOCAP, and MELD, our method outperforms prompting and finetuning baselines on both emotion determination and response quality. We finally implement an entire streaming pipeline for real-time deployment, with the speech quality confirming the framework's emotional alignment, contextual coherence, and expressive fluency. Codes, cases, and demos are available in https://sixingdeguo.github.io/EmoQ-page/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。