提出统一建模音色与节奏的自适应语音合成方法，提升未知场景下语音自然度。
AS-Speech: Adaptive Style For Speech Synthesis
- 通过细粒度文本驱动音色特征与全局节奏信息融合建模
- 在多个评估指标上优于现有自适应TTS模型，音色与节奏相似性更高
- 适合需要高保真语音合成的个性化语音应用
近年来，文本到语音（TTS）合成技术取得显著进展，可在常见场景下实现高质量语音合成。但在未见场景中，自适应TTS需具备强泛化能力以应对说话人风格变化。然而，现有自适应方法仅能分别提取和整合粗粒度音色或混合节奏属性。本文提出AS-Speech，一种将说话人音色特征与节奏属性统一建模的自适应风格方法。具体而言，AS-Speech通过细粒度文本驱动的音色特征与全局节奏信息，精确模拟风格特征，并借助扩散模型实现高保真语音合成。实验表明，所提模型在音色与节奏的自然度和相似性方面均优于一系列自适应TTS模型。
原文摘要 · Abstract (English)
In recent years, there has been significant progress in Text-to-Speech (TTS) synthesis technology, enabling the high-quality synthesis of voices in common scenarios. In unseen situations, adaptive TTS requires a strong generalization capability to speaker style characteristics. However, the existing adaptive methods can only extract and integrate coarse-grained timbre or mixed rhythm attributes separately. In this paper, we propose AS-Speech, an adaptive style methodology that integrates the speaker timbre characteristics and rhythmic attributes into a unified framework for text-to-speech synthesis. Specifically, AS-Speech can accurately simulate style characteristics through fine-grained text-based timbre features and global rhythm information, and achieve high-fidelity speech synthesis through the diffusion model. Experiments show that the proposed model produces voices with higher naturalness and similarity in terms of timbre and rhythm compared to a series of adaptive TTS models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。