用神经转换器实现端到端语音合成，无需显式预测音长。
TTS-Transducer: End-to-End Speech Synthesis with Neural Transducer
- 采用转换器学习文本与声码器码本的单调对齐
- 首码本通过转换器建模，其余码本用非自回归变换器预测
- 端到端训练，兼具高质量与强鲁棒性，适合语音生成研究
本文提出TTS-Transducer，一种新型端到端文本到语音合成架构，结合音频码本模型与神经转换器优势。转换器擅长学习单调对齐，避免使用显式音长预测器；神经音频码本可高效压缩音频为离散码本，使文本建模方法可用于语音生成。然而，残差量化码本需每帧预测多个码本，带来显著挑战。该系统首先利用转换器对齐文本与首个码本的语音码本，随后通过非自回归Transformer基于转换器损失提取的对齐，预测其余码本。系统全程端到端训练。实验表明，TTS-Transducer在质量与鲁棒性上媲美主流语音合成系统，是一种有竞争力且稳定的替代方案。
原文摘要 · Abstract (English)
This work introduces TTS-Transducer - a novel architecture for text-to-speech, leveraging the strengths of audio codec models and neural transducers. Transducers, renowned for their superior quality and robustness in speech recognition, are employed to learn monotonic alignments and allow for avoiding using explicit duration predictors. Neural audio codecs efficiently compress audio into discrete codes, revealing the possibility of applying text modeling approaches to speech generation. However, the complexity of predicting multiple tokens per frame from several codebooks, as necessitated by audio codec models with residual quantizers, poses a significant challenge. The proposed system first uses a transducer architecture to learn monotonic alignments between tokenized text and speech codec tokens for the first codebook. Next, a non-autoregressive Transformer predicts the remaining codes using the alignment extracted from transducer loss. The proposed system is trained end-to-end. We show that TTS-Transducer is a competitive and robust alternative to contemporary TTS systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。