arXiv:2607.03928cs.SDcs.AI2026-07

用自监督语音符号实现无合成数据的口音标准化,保持说话人身份。

TokAN: Accent Normalization Using Self-Supervised Speech Tokens

论文配图:TokAN: Accent Normalization Using Self-Supervised Speech Tokens
图 1 · 摘自论文原文
  • 基于自监督离散语音符号,通过自回归模型转换口音序列
  • 词错误率从12.40%降至9.23%,显著提升可懂度
  • 适合配音、直播等需精准时长控制的场景

口音标准化(AN)旨在将非母语(L2)口音语音转化为标准母语(L1)语音,同时保留说话人特征。现有方法或需真实平行的L1-L2语音训练,或在使用合成目标时导致质量下降。本文提出TokAN,一种基于自监督离散语音符号的口音标准化框架,该符号由联合训练的L1-L2向量量化(VQ)分词器提取,无需合成监督语音。一个自回归编码器-解码器模型执行符号到符号的转换,将带有L2口音的符号序列转为标准语音符号。我们还引入基于组相对策略优化(GRPO)的强化学习后训练,以词错误率和口音分类器置信度作为互补奖励。一个非自回归流匹配声谱恢复器根据源说话人嵌入还原梅尔频谱。此外,我们设计了支持总时长感知的流匹配时长预测器,使TokAN适用于配音和实时播音等对时长敏感的任务。在七种英语口音上的实验表明,经监督微调后,字错误率从12.40%降至9.89%,强化学习后进一步降至9.23%,始终优于帧对帧、直接流匹配及提示式符号转换基线,在降低口音和提升可懂度方面表现更优。

原文摘要 · Abstract (English)

Accent normalization (AN) seeks to convert non-native (L2) accented speech into standard (L1) speech while preserving speaker identity. The current techniques either require naturally recorded parallel L1-L2 speech for training, or suffer from quality degradation when supervised by synthesized targets. In this paper, we present TokAN, a token-based accent normalization framework that operates on self-supervised discrete speech tokens extracted from a L1-L2 jointly trained vector-quantization (VQ) tokenizer, without the need of synthetic supervisory speech. An autoregressive encoder-decoder model performs token-to-token conversion, translating L2-accented token sequences into the tokens of standard voice. We also introduce reinforcement learning (RL) post-training based on Group Relative Policy Optimization (GRPO), using word error rate and accent classifier confidence as complementary rewards. A non-autoregressive flow-matching synthesizer recovers the Mel-spectrogram from the converted tokens, conditioned on the source speaker embedding. We also develop a flow-matching duration predictor that supports total-duration-aware synthesis, making TokAN applicable to duration-critical tasks such as voice dubbing and live casting. Experiments on seven English accents demonstrate that TokAN reduced the word error rate from 12.40% to 9.89% after supervised fine-tuning, and further to 9.23% after RL post-training, consistently outperforming frame-to-frame, direct flow-matching, and prompt-based token-conversion baselines in terms of accent reduction and intelligibility.

口音标准化自监督语音合成强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。