用大模型+语音克隆打造可对话的真人音色聊天机器人
JoyTTS: LLM-based Spoken Chatbot With Voice Cloning
- 结合大语言模型与声纹克隆技术，实现端到端语音对话
- 在测试机上达到0.73的说话人相似度和5.09%的词错误率
- 开源完整代码与模型，适合语音交互与个性化应用研究
JoyTTS 是一个端到端的语音聊天机器人，融合大语言模型（LLM）与文本转语音（TTS）技术，具备语音克隆能力。该项目基于开源的 MiniCPM-o 与 CosyVoice2 模型，在 2000 小时的对话数据上进行训练。测试机 seed-tts-zh 上，其说话人相似度（SS）得分为 0.73，词错误率（WER）为 5.09。项目已公开全部训练代码，支持社区进一步开发与优化。代码、模型及训练推理脚本详见 https://github.com/jdh-algo/JoyTTS.git。
原文摘要 · Abstract (English)
JoyTTS is an end-to-end spoken chatbot that combines large language models (LLM) with text-to-speech (TTS) technology, featuring voice cloning capabilities. This project is built upon the open-source MiniCPM-o and CosyVoice2 models and trained on 2000 hours of conversational data. We have also provided the complete training code to facilitate further development and optimization by the community. On the testing machine seed-tts-zh, it achieves a SS (speaker similarity) score of 0.73 and a WER (Word Error Rate) of 5.09. The code and models, along with training and inference scripts, are available at https://github.com/jdh-algo/JoyTTS.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。