MahaTTSv2让语音合成支持印度语系，用2万小时数据训练，效果优于现有模型。
MahaTTS: A Unified Framework for Multilingual Text-to-Speech Synthesis
- 用Wav2Vec2.0提取语义，语言模型建模文本到语义映射。
- 在2万小时印度语数据上训练，支持多说话人、多语言语音合成。
- 适合关注低资源语言语音合成的研究者与开发者。
当前语音合成模型多聚焦英语和欧洲语言，难以覆盖更多人群。为此，我们提出MahaTTS-v2——一个支持多说话人、多语言的语音合成系统，特别强化了印地语等印度语言的表现力。模型基于约2万小时的印度语语音数据进行训练。方法上，采用Wav2Vec2.0提取语义特征，并通过语言模型实现文本到语义的建模；再利用条件流模型（CFM）将语义转换为梅尔频谱。实验表明，该框架在多语言合成任务中优于现有方法。代码已开源：https://github.com/dubverse-ai/MahaTTSv2。
原文摘要 · Abstract (English)
Current Text-to-Speech models pose a multilingual challenge, where most of the models traditionally focus on English and European languages, thereby hurting the potential to provide access to information to many more people. To address this gap, we introduce MahaTTS-v2 a Multilingual Multi-speaker Text-To-Speech (TTS) system that has excellent multilingual expressive capabilities in Indic languages. The model has been trained on around 20K hours of data specifically focused on Indian languages. Our approach leverages Wav2Vec2.0 tokens for semantic extraction, and a Language Model (LM) for text-to-semantic modeling. Additionally, we have used a Conditional Flow Model (CFM) for semantics to melspectogram generation. The experimental results indicate the effectiveness of the proposed approach over other frameworks. Our code is available at https://github.com/dubverse-ai/MahaTTSv2
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。