用自然语言控制语音情绪，让合成语音更真实灵活。
PromptEVC: Controllable Emotional Voice Conversion with Natural Language Prompts
- 用自然语言描述情绪，自动生成精细情感嵌入。
- 在多个指标上优于现有方法，支持混合情绪和节奏调控。
- 适合语音合成、情感计算与交互式语音系统研究者。
可控情感语音转换旨在通过调节情感表达提升合成语音的多样性。现有方法通常依赖预定义标签、参考音频或指定参数值，忽视了个体对情绪感知与表达的差异。本文提出 PromptEVC，利用自然语言提示实现精确且灵活的情感控制。为连接文本描述与情感语音，我们设计情感描述符与提示映射器，生成细粒度情感嵌入，并与参考嵌入联合训练。为提升自然度，提出韵律建模与控制流程，根据语言内容和情感线索调整节奏。同时引入说话人编码器以保留身份特征。实验表明，PromptEVC 在情感转换、强度控制、混合情感合成及韵律调节方面均优于当前最优方法。语音样例可访问 https://jeremychee4.github.io/PromptEVC/。
原文摘要 · Abstract (English)
Controllable emotional voice conversion (EVC) aims to manipulate emotional expressions to increase the diversity of synthesized speech. Existing methods typically rely on predefined labels, reference audios, or prespecified factor values, often overlooking individual differences in emotion perception and expression. In this paper, we introduce PromptEVC that utilizes natural language prompts for precise and flexible emotion control. To bridge text descriptions with emotional speech, we propose emotion descriptor and prompt mapper to generate fine-grained emotion embeddings, trained jointly with reference embeddings. To enhance naturalness, we present a prosody modeling and control pipeline that adjusts the rhythm based on linguistic content and emotional cues. Additionally, a speaker encoder is incorporated to preserve identity. Experimental results demonstrate that PromptEVC outperforms state-of-the-art controllable EVC methods in emotion conversion, intensity control, mixed emotion synthesis, and prosody manipulation. Speech samples are available at https://jeremychee4.github.io/PromptEVC/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。