首个实时开源对话模型，能精准克隆个性化语音。
FlashLabs Chroma 1.0: A Real-Time End-to-End Spoken Dialogue Model with Personalized Voice Cloning
- 采用1:2交错文本音频令牌调度，实现低延迟流式生成。
- 语音相似度比真人基准提升10.96%，实时因子0.43。
- 适合需要高保真语音克隆的智能客服与虚拟助手场景。
近期端到端语音对话系统利用语音标记器和神经音频编码器，使大语言模型可直接处理离散语音表示。然而，这些模型常表现出说话人身份保留能力有限的问题，阻碍个性化语音交互。本文提出Chroma 1.0，首个开源、实时、端到端的语音对话模型，兼具低延迟交互与高保真个性化语音克隆能力。Chroma通过交错文本-音频令牌调度（1:2）实现亚秒级端到端延迟，支持流式生成，同时在多轮对话中保持高质量个性化语音合成。实验结果表明，Chroma在说话人相似度上相较人类基线提升10.96%，实时因子（RTF）为0.43，且维持强大的推理与对话能力。代码与模型已公开于https://github.com/FlashLabs-AI-Corp/FlashLabs-Chroma 和 https://huggingface.co/FlashLabs/Chroma-4B。
原文摘要 · Abstract (English)
Recent end-to-end spoken dialogue systems leverage speech tokenizers and neural audio codecs to enable LLMs to operate directly on discrete speech representations. However, these models often exhibit limited speaker identity preservation, hindering personalized voice interaction. In this work, we present Chroma 1.0, the first open-source, real-time, end-to-end spoken dialogue model that achieves both low-latency interaction and high-fidelity personalized voice cloning. Chroma achieves sub-second end-to-end latency through an interleaved text-audio token schedule (1:2) that supports streaming generation, while maintaining high-quality personalized voice synthesis across multi-turn conversations. Our experimental results demonstrate that Chroma achieves a 10.96% relative improvement in speaker similarity over the human baseline, with a Real-Time Factor (RTF) of 0.43, while maintaining strong reasoning and dialogue capabilities. Our code and models are publicly available at https://github.com/FlashLabs-AI-Corp/FlashLabs-Chroma and https://huggingface.co/FlashLabs/Chroma-4B .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。