解决多语言语音合成中语码转换不自然的问题
SFMS-ALR: Script-First Multilingual Speech Synthesis with Adaptive Locale Resolution
- 先按字符集分段,再自适应识别语言和语境
- 通过情感感知调整韵律,保持跨语言表达连贯
- 无需重训练,可直接对接主流语音引擎
句内多语言语音合成(语码转换TTS)因语言突变、字符差异和韵律不匹配仍面临重大挑战。传统TTS系统多为单语种,难以在混合语言场景下生成自然可懂的语音。本文提出一种与引擎无关的脚本优先多语言合成框架——SFMS-ALR,实现流畅实时的语码转换语音生成。该框架首先按Unicode字符集对输入文本进行分段,利用自适应语言识别确定每段的语言及语境,通过情感感知的韵律归一化技术保持跨语言表达连续性。算法生成统一的SSML表示,包含适当的"lang"或"voice"标签,并以单次TTS请求完成合成。相较于端到端多语言模型,SFMS-ALR无需重新训练,可无缝集成谷歌、苹果、亚马逊等平台现有语音资源。与Unicom、Mask LID等数据驱动流水线相比,其具有更高的灵活性、可解释性和即插即用能力。该框架为高质量、引擎独立的多语言TTS建立模块化基准,并提出可理解性、自然度和用户偏好等评估策略。
原文摘要 · Abstract (English)
Intra-sentence multilingual speech synthesis (code-switching TTS) remains a major challenge due to abrupt language shifts, varied scripts, and mismatched prosody between languages. Conventional TTS systems are typically monolingual and fail to produce natural, intelligible speech in mixed-language contexts. We introduce Script-First Multilingual Synthesis with Adaptive Locale Resolution (SFMS-ALR), an engine-agnostic framework for fluent, real-time code-switched speech generation. SFMS-ALR segments input text by Unicode script, applies adaptive language identification to determine each segment's language and locale, and normalizes prosody using sentiment-aware adjustments to preserve expressive continuity across languages. The algorithm generates a unified SSML representation with appropriate "lang" or "voice" spans and synthesizes the utterance in a single TTS request. Unlike end-to-end multilingual models, SFMS-ALR requires no retraining and integrates seamlessly with existing voices from Google, Apple, Amazon, and other providers. Comparative analysis with data-driven pipelines such as Unicom and Mask LID demonstrates SFMS-ALR's flexibility, interpretability, and immediate deployability. The framework establishes a modular baseline for high-quality, engine-independent multilingual TTS and outlines evaluation strategies for intelligibility, naturalness, and user preference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。