轻量级适配器让F5-TTS支持罗马尼亚语语音合成
F5-TTS-RO: Extending F5-TTS to Romanian TTS via Lightweight Input Adaptation
- 冻结原模型权重，仅新增子网络适配罗马尼亚语字符嵌入
- 人类听者评测显示语音自然度高，可实现中英混用发音
- 适合需要快速扩展多语言支持的语音合成研究者
本文提出一种轻量级输入层适配器，使F5-TTS模型支持罗马尼亚语语音合成。为保留原有功能（语音克隆、英语和中文支持），我们冻结原始权重，在文本编码器的词嵌入矩阵后添加子网络进行训练。采用F5-TTS中的ConvNeXt模块建模新字符嵌入间的依赖关系，作为‘软’音素映射层，将罗马尼亚语文本转换为连续表示以生成自然语音。通过20名听者在三项任务上的评估：（a）参考语音与生成语音的相似性，（b）发音准确性和自然度，（c）罗马尼亚语-英语代码切换能力。结果表明该方法保持了语音克隆能力，并在一定程度上实现单句内混用语言；但仍有残留英文口音。代码与示例音频已开源至https://github.com/racai-ro/Ro-F5TTS。
原文摘要 · Abstract (English)
This work introduces a lightweight input-level adapter for the F5-TTS model that enables Romanian Language support. To preserve the existing capabilities of the model (voice cloning, English and Chinese support), we keep the original weights frozen, append a sub-network to the model and train it as an extension for the textual embedding matrix of the text encoder. For simplicity, we rely on ConvNeXt module implemented in F5-TTS to also model the co-dependencies between the new character-level embeddings. The module serves as a ``soft`` letter-to-sound layer, converting Romanian text into a continuous representation that the F5-TTS model uses to produce naturally sounding Romanian utterances. We evaluate the model with a pool of 20 human listeners across three tasks: (a) audio similarity between reference and generated speech, (b) pronunciation and naturalness and (c) Romanian-English code-switching. The results indicate that our approach maintains voice cloning capabilities and enables, to a certain extent, code-switching within the same utterance; however, residual English accent characteristics remain. We open-source our code and provide example audio samples at https://github.com/racai-ro/Ro-F5TTS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。