开源语音合成系统支持自然语言指令控制，可生成多说话人多轮对话。
Fish Audio S2 Technical Report
- 分阶段训练+数据流水线，提升多说话人语音合成效率。
- 实时语音生成延迟低于100毫秒，实时因子达0.195。
- 适合需要自定义语音、快速部署的开发者和研究者使用。
我们提出 Fish Audio S2，一个开源文本到语音系统，具备多说话人、多轮生成能力，最重要的是可通过自然语言描述实现指令跟随控制。为提升训练规模，我们设计了多阶段训练方案与分阶段数据流水线，涵盖视频字幕、语音字幕、音质评估及奖励建模。为推动开源语音合成发展，我们发布了模型权重、微调代码以及基于 SGLang 的推理引擎。该引擎支持流式传输，实现 0.195 的实时因子（RTF）和低于 100 毫秒的首音频延迟。代码与模型权重已公开于 GitHub（https://github.com/fishaudio/fish-speech）和 Hugging Face（https://huggingface.co/fishaudio/s2-pro）。强烈推荐访问 https://fish.audio 体验自定义语音。
原文摘要 · Abstract (English)
We introduce Fish Audio S2, an open-sourced text-to-speech system featuring multi-speaker, multi-turn generation, and, most importantly, instruction-following control via natural-language descriptions. To scale training, we develop a multi-stage training recipe together with a staged data pipeline covering video captioning and speech captioning, voice-quality assessment, and reward modeling. To push the frontier of open-source TTS, we release our model weights, fine-tuning code, and an SGLang-based inference engine. The inference engine is production-ready for streaming, achieving an RTF of 0.195 and a time-to-first-audio below 100 ms.Our code and weights are available on GitHub (https://github.com/fishaudio/fish-speech) and Hugging Face (https://huggingface.co/fishaudio/s2-pro). We highly encourage readers to visit https://fish.audio to try custom voices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。