arXiv:2607.09530cs.CL2026-07

轻量级土耳其语语音合成模型,无需分词器即可高效生成自然语音。

FreyaTTS: A Compact Tokenizer-Free Flow-Matching Transformer for Turkish-First Speech Synthesis

  • 直接用字符输入,无分词或音素转换,端到端生成语音。
  • 183万参数,实时生成,错误率低于更大模型。
  • 适合边缘设备部署,开源可用,语音自然度高。

我们提出 Freya-TTS,一种专为土耳其语设计的紧凑、免分词、基于流匹配的非自回归语音合成模型。该模型为183.2M参数的条件流匹配扩散变换器(DiT),在冻结的AudioVAE2连续潜空间中运行(编码16 kHz,解码48 kHz),使模型专注于文本到潜空间映射,同时继承高质量48 kHz重建能力。其创新体现在三方面:(1)无需规则的端到端建模,使用92个土耳其字符词汇表,不依赖音素化、音素转写或离散语音分词器,数字串在前端扩展为口语形式;(2)非自回归并行去噪,基于预测时长一次性生成完整潜序列;(3)面向生产环境的两阶段后训练策略,包含单说话人锁定与短句覆盖,提升说话人一致性与短输入鲁棒性。在Freya-TR-Eval基准上,模型达到8.0%的带宽匹配词错误率(WER)和3.0%的字符错误率(CER),低于同领域更大模型XTTS-v2和F5-TTS(参数量仅为后者的40-55%),且在紧凑系统中自然度评分(MOS)最高。模型在消费级显卡RTX 4090上实现实时因子0.11(平均0.14于H100),可在笔记本CPU上实时生成,适用于资源受限的边缘部署。模型权重、训练与推理代码及评估基准已开源,采用Apache-2.0许可。

原文摘要 · Abstract (English)

We introduce Freya-TTS, a compact, tokenizer-free, Turkish-first text-to-speech model designed for highly reliable and efficient conversational synthesis. Freya-TTS is a 183.2M-parameter non-autoregressive conditional flow-matching Diffusion Transformer (DiT) that operates in the continuous latent space of the frozen AudioVAE2 (16 kHz encode, 48 kHz decode), allowing the model to focus its capacity on text-to-latent mapping while inheriting high-quality 48 kHz reconstruction. We advance the framework along three key dimensions: (1) rule-free end-to-end modeling from a 92-symbol Turkish character vocabulary without a phonemizer, grapheme-to-phoneme frontend, or discrete speech tokenizer, with digit strings expanded to their spoken form at the text frontend; (2) non-autoregressive parallel denoising, which predicts the entire latent sequence simultaneously over a predicted duration; and (3) a production-oriented two-stage post-training recipe consisting of single-speaker voice locking and short-utterance coverage, improving speaker consistency and robustness on short inputs. On the Freya-TR-Eval benchmark, Freya-TTS achieves a band-matched word error rate (WER) of 8.0% and character error rate (CER) of 3.0%, lower error than both larger open systems in its field, XTTS-v2 and F5-TTS, at 40-55% of their parameter count, together with the highest naturalness (MOS) among the compact systems. The model achieves a real-time factor of 0.11 on a consumer GPU (RTX 4090; ~0.14 mean on an H100) and synthesizes in real time on a laptop CPU, making it well suited for resource-constrained edge deployment. We release the model weights, training and inference code, and evaluation benchmark under the Apache-2.0 license.

语音合成轻量化土耳其语扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。