用文本提示实现高效稳定的语音风格控制,无需参考音频或大模型。
ParaStyleTTS: Toward Efficient and Robust Paralinguistic Style Control for Expressive Text-to-Speech Generation
- 分两级建模语调与副语言风格,实现细粒度控制。
- 速度比顶尖大模型快30倍,参数少8倍,显存需求降2.5倍。
- 适合手机端等资源受限场景,对提示词不敏感,实用性强。
在文本转语音系统中控制说话风格已成为学术界和产业界关注的焦点。现有方法多依赖参考音频引导风格生成,但存在隐私问题且难以获取;近期基于大语言模型(LLMs)的方法虽可通过自然语言提示控制风格,却因计算开销高、可解释性差及对提示敏感,难以应用于实时或资源受限环境。本文提出ParaStyleTTS,一种轻量且可解释的语音合成框架,仅通过文本提示即可实现表达性风格控制。该框架采用新颖的两层风格适配架构,分离语调与副语言风格建模,可精细调控情绪、性别、年龄等因子。相比LLM方法,ParaStyleTTS在不同提示表述下保持一致的风格输出,更适合实际应用,包括设备端与低资源部署。实验表明,其生成语音质量媲美当前最先进方法,同时速度提升30倍,参数量减少8倍,CUDA内存占用降低2.5倍。此外,其在副语言风格控制上展现出更强鲁棒性与可控性,为风格可控语音合成提供了一种高效实用的解决方案。
原文摘要 · Abstract (English)
Controlling speaking style in text-to-speech (TTS) systems has become a growing focus in both academia and industry. While many existing approaches rely on reference audio to guide style generation, such methods are often impractical due to privacy concerns and limited accessibility. More recently, large language models (LLMs) have been used to control speaking style through natural language prompts; however, their high computational cost, lack of interpretability, and sensitivity to prompt phrasing limit their applicability in real-time and resource-constrained environments. In this work, we propose ParaStyleTTS, a lightweight and interpretable TTS framework that enables expressive style control from text prompts alone. ParaStyleTTS features a novel two-level style adaptation architecture that separates prosodic and paralinguistic speech style modeling. It allows fine-grained and robust control over factors such as emotion, gender, and age. Unlike LLM-based methods, ParaStyleTTS maintains consistent style realization across varied prompt formulations and is well-suited for real-world applications, including on-device and low-resource deployment. Experimental results show that ParaStyleTTS generates high-quality speech with performance comparable to state-of-the-art LLM-based systems while being 30x faster, using 8x fewer parameters, and requiring 2.5x less CUDA memory. Moreover, ParaStyleTTS exhibits superior robustness and controllability over paralinguistic speaking styles, providing a practical and efficient solution for style-controllable text-to-speech generation. Demo can be found at https://parastyletts.github.io/ParaStyleTTS_Demo/. Code can be found at https://github.com/haoweilou/ParaStyleTTS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。