用语音定位自动识别口音，提升多口音语音合成的规模与质量。
Scalable Controllable Accented TTS
- 通过语音地理定位自动推断口音标签，减少人工标注依赖。
- 利用kNN声线转换增强数据多样性，提升模型鲁棒性。
- 在CommonVoice上验证，效果优于现有基准和自报口音数据训练的模型。
本文解决多口音语音合成系统扩展难题，使其能处理更大规模训练数据及更广泛口音标签，包括传统数据集中代表性不足或未标注的口音。为此提出两种策略：1. 基于语音地理定位模型，从原始语音中自动推断口音标签，无需依赖人工标注；2. 通过kNN语音转换进行声线增强，提升数据多样性与模型鲁棒性。在CommonVoice数据集上，对XTTS-v2进行微调，使用地理定位发现或增强的口音标签。结果表明，该模型不仅优于基于用户自报口音标签微调的XTTS-v2，还超越现有口音语音合成基准。
原文摘要 · Abstract (English)
We tackle the challenge of scaling accented TTS systems, expanding their capabilities to include much larger amounts of training data and a wider variety of accent labels, even for accents that are poorly represented or unlabeled in traditional TTS datasets. To achieve this, we employ two strategies: 1. Accent label discovery via a speech geolocation model, which automatically infers accent labels from raw speech data without relying solely on human annotation; 2. Timbre augmentation through kNN voice conversion to increase data diversity and model robustness. These strategies are validated on CommonVoice, where we fine-tune XTTS-v2 for accented TTS with accent labels discovered or enhanced using geolocation. We demonstrate that the resulting accented TTS model not only outperforms XTTS-v2 fine-tuned on self-reported accent labels in CommonVoice, but also existing accented TTS benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。