用两个参考音频实现语音风格与音色的独立精准控制。
FC-TTS: Style and Timbre Control in Zero-Shot Text-to-Speech with Disentangled Speech Representations

- 通过双参考输入与解耦表征,分离语音的风格与音色属性。
- 合成语音保真度高,零样本自然度表现优异。
- 适合需要精细语音控制的应用场景,如语音克隆与角色配音。
零样本文本到语音(TTS)技术已能准确模仿参考语音的说话风格和音色,但实现两者独立可控仍具挑战。已有研究提出解耦语音表征,将语音分解为可解释属性(如音色、语调、内容),为属性控制提供了基础。然而,如何有效将这些表征融入TTS系统以实现独立精确控制仍不明确。本文提出FC-TTS框架,通过两个不同参考语音实现风格与音色的解耦控制。不同于依赖预训练解耦表征的现有方法,FC-TTS引入关键设计策略,包括架构选择、训练框架和辅助目标函数,提升了属性分离的可靠性与双参考控制能力。实验表明,FC-TTS在高质量语音合成和零样本自然度方面表现优异,且可稳定独立地操控风格与音色。音频样例见:https://qualcomm-ai-research.github.io/fc-tts
原文摘要 · Abstract (English)
Recent advances in zero-shot text-to-speech (TTS) have enabled accurate imitation of reference speech in terms of both speaking style and speaker timbre. However, achieving disentangled control over these aspects from separate references remains a challenging task. Several studies have proposed disentangled speech representations that decompose speech into interpretable attributes (e.g., timbre, prosody, and content), providing a promising foundation for TTS with attribute control from separate references. Yet, how to effectively integrate such representations into TTS systems to achieve independent and precise control remains underexplored. In this paper, we present FC-TTS, a zero-shot TTS framework that enables disentangled control of style and timbre by conditioning on two distinct reference utterances. Unlike existing systems that inherit limitations from those pre-trained disentangled representations, FC-TTS introduces key design strategies, including architectural choices, training framework, and auxiliary training objectives, which improve the reliability of attribute separation and dual-reference control. Experiments show that FC-TTS achieves high-fidelity synthesis and competitive zero-shot naturalness, while uniquely supporting consistent and independent manipulation of style and timbre. Audio samples are available at https://qualcomm-ai-research.github.io/fc-tts
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。