用连续语音标记提升语音合成自然度与保真度
Continuous Speech Tokenizer in Text To Speech
- 提出连续语音标记方法Cont-SPT，替代传统离散标记
- 在高低频段均实现更高信息保留率，提升语音连贯性
- 适合追求高保真语音合成的开发者和研究者
大语言模型时代下，语音与语言的融合备受关注。文本到语音任务中常采用离散语音标记以实现语音压缩和可移植性，便于与文本联合训练且压缩效率高。然而我们发现，离散语音标记仍存在信息损失问题。为此，提出一种简单有效的连续语音标记方法Cont-SPT，以及基于连续语音标记的文语合成模型。结果表明，基于连续语音标记的语音语言模型在语音连续性与主观评分（平均意见分，MoS）上表现更优。这一提升源于连续语音标记在频域中对高低频信息的更好保留能力。代码与资源详见 https://github.com/Yixing-Li/Continuous-Speech-Tokenizer。
原文摘要 · Abstract (English)
The fusion of speech and language in the era of large language models has garnered significant attention. Discrete speech token is often utilized in text-to-speech tasks for speech compression and portability, which is convenient for joint training with text and have good compression efficiency. However, we found that the discrete speech tokenizer still suffers from information loss. Therefore, we propose a simple yet effective continuous speech tokenizer named Cont-SPT, and a text-to-speech model based on continuous speech tokens. Our results show that the speech language model based on the continuous speech tokenizer has better continuity and higher estimated Mean Opinion Scores (MoS). This enhancement is attributed to better information preservation rate of the continuous speech tokenizer across both low and high frequencies in the frequency domain. The code and resources for Cont-SPT can be found in https://github.com/Yixing-Li/Continuous-Speech-Tokenizer
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。