arXiv:2606.05367cs.SDeess.AS2026-06

通过消除实验定位情感语调载体,实现无需训练的情感控制。

Task-Vector Arithmetic for Emotional Expressivity Control in Language-Model-Based Text-to-Speech

  • 用消融实验确定情感语调主要由联合训练的说话人嵌入承载。
  • 跨语言迁移下情感相似度提升0.29(英语)和0.09(葡萄牙语)。
  • 无需微调,可直接通过向量运算调节情感强度,适合快速部署。

我们研究任务向量算术在基于语言模型的端到端语音合成(LM-TTS)中是否适用于跨说话人情感强度控制。通过对Qwen3-TTS-12Hz-1.7B模型进行系统性消融实验,分别考察LoRA微调、连续编码器嵌入、离散编码器标记以及由联合训练的ECAPA-TDNN编码器生成的说话人嵌入(x-vector)的影响,发现情感语调的主要载体是x-vector。基于此,提出一种免训练方法:在x-vector空间中计算情感方向τ = 𝔼ᵢ[x(sᵢ, emo)] − 𝔼ᵢ[x(sᵢ, neutral)],并应用于目标说话人:x_new = x(target, neutral) + α·τ。以ESD(英语)为τ源,emoUERJ(巴西葡萄牙语)为跨语言验证集,结果表明在英语未见说话人上情感相似度(emotion2vec cosine)平均提升+0.29,在巴西葡萄牙语未见说话人上提升+0.09;同时身份保留良好(WavLM SECS ≳ 0.88),语音可懂度高(WER ≈ 0)。这些结果首次提供证据表明该类模型中的情感语调载体可通过消融定位至联合训练的说话人嵌入,且免训练的质心算术在跨语言场景下仍有效。

原文摘要 · Abstract (English)

We investigate whether task-vector arithmetic, successful for cross-speaker emotional intensity control in modular text-to-speech (TTS), transfers to large-scale TTS systems built on language-model backbones with in-context learning (LM-TTS). Through a systematic elimination study over four progressively narrower operands on Qwen3-TTS-12Hz-1.7B - model weights via LoRA fine-tuning, continuous codec embeddings, discrete codec tokens, and the speaker embedding (x-vector) produced by an ECAPA-TDNN encoder jointly trained with the synthesis backbone - we localize the dominant carrier of emotional prosody to the x-vector. Building on this finding, we propose a training-free method based on centroid arithmetic in x-vector space: an emotion direction $τ= \mathbb{E}_i[x(s_i,\text{emo})] -\mathbb{E}_i[x(s_i,\text{neutral})]$ applied to an unseen target speaker as $x_{\text{new}} = x(\text{target},\text{neutral}) + α\cdotτ$. Using ESD (English) as the $τ$ source and emoUERJ (Brazilian Portuguese) as a cross-lingual ground-truth target, we observe average gains of $+0.29$ in emotion2vec cosine over the ICL baseline on English held-out speakers and $+0.09$ on Brazilian Portuguese held-out speakers, while largely preserving identity (WavLM SECS $\gtrsim 0.88$ for the multi-speaker $τ$ variant) and intelligibility (WER $\approx 0$ in PT-BR). These results offer initial evidence that the dominant carrier of emotional prosody in this class of models is localizable, by elimination, to the co-trained speaker embedding, where training-free centroid arithmetic remains effective even under cross-lingual transfer.

语音合成情感控制向量算术零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。