让语音合成能随语句精细调节风格，支持跨句渐变和句内动态切换。
Unlocking Fine-Grained and Within-Utterance Speaking Style Control in Prompt-Based Text-to-Speech Models
- 用提示向量插值实现跨句子风格平滑过渡
- 通过缓存替换与注意力掩码，解决早期音素主导问题
- 支持语音风格在句子内连续变化，适合影视配音等场景
基于提示的文本到语音模型虽能通过自然语言控制发音风格，但通常缺乏细粒度调控能力，且整句话采用单一全局风格，限制了需要跨句连续风格插值和句内风格动态变化的实际应用。本文提出新方法，在现有提示式TTS模型中同时实现这两种能力。对于跨句风格插值，通过计算对比风格提示在嵌入空间中的方向向量并进行简单插值，实现风格特征的平滑过渡。针对句内风格变化，发现自回归解码器存在对早期词元的强注意力偏倚，导致初始音频生成主导后续内容。为此引入KV缓存交换与滑动窗口注意力掩码机制以缓解此问题。实验表明，跨句插值在性别转换上成功率达99%-100%，可实现最高36 Hz的音高变化和最高1.6音节/秒的速度调整；句内风格转换保持说话人相似度0.81-0.91，感知流畅度评分达3.48-4.48。
原文摘要 · Abstract (English)
While prompt-based text-to-speech (TTS) models enable natural language-driven speaking style control, they often provide limited fine-grained control and apply a single global style across an utterance. This restricts practical use cases that require continuous style attribute interpolation across utterances and time-varying style transitions within a single utterance. In this paper, we propose novel techniques to achieve both capabilities in existing prompt-based TTS models. For inter-utterance style interpolation, we compute direction vectors between contrastive style prompts in the embedding space and perform simple interpolation, enabling smooth transitions between style characteristics. For intra-utterance style transition, we first identify a strong attention bias toward early tokens in autoregressive TTS decoders, causing the initial audio realization to dominate subsequent generation. To mitigate this effect, we introduce KV-cache swapping and sliding-window attention masking. Experiments demonstrate that our proposed inter-utterance interpolation achieves a 99-100% success rate in gender conversion, up to 36 Hz pitch variation, and up to 1.6 syllables-per-second speed change. Our intra-utterance transition maintains a speaker similarity of 0.81-0.91 and achieves perceptual smoothness scores of 3.48-4.48.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。