让语音克隆模型在不同文字间保持同一说话人身份，提升跨语言语音合成效果。
LASE: Language-Adversarial Speaker Encoding for Indic Cross-Script Identity Preservation
- 用对抗性损失让语音编码器忽略语言信息，只保留说话人特征。
- 在西印和印式语料上，跨文字相似度差距接近零，优于基线模型2.4~2.7倍。
- 仅需少量数据即可达到顶级性能，适合资源受限的语音应用开发。
多语言语音克隆中的说话人编码器应无视语音所用文字，对同一说话人保持一致表示。现成编码器存在此缺陷，且影响随口音而异。在包含英语、印地语、泰卢固语和泰米尔语的1043对西方口音语料上，WavLM-base-plus-sv在跨文字时余弦相似度下降0.082，ECAPA-TDNN下降0.105；在1369对印度口音语料上，差距缩小至0.006（WavLM-SV）和0.044（ECAPA-TDNN）。该问题在非印地语训练系统向印地语文字迁移时最为显著。本文提出LASE（语言对抗性说话人编码器），在冻结的WavLM-base-plus基础上添加一个小投影头，通过两个损失训练：监督对比损失用于保留说话人身份，梯度反转交叉熵损失则对抗一个四语言分类器，使嵌入尽可能不携带语言信息但仍保留说话人信息。在1118个经质量筛选的跨文字语音对上训练，其残差差距在两种语料上均接近零（西印Δ=0.013，印式Δ=0.026；两者95%置信区间均含零），并使跨文字与基准的差距扩大2.4~2.7倍。消融实验表明，梯度反转损失提升任一主干网络，但选择WavLM本身也贡献显著。在合成多说话人语音分离任务中，LASE以约100倍更少的训练数据达到了与ECAPA-TDNN相当的跨文字说话人召回率（0.788 vs 0.789）。本文发布r1检查点、两个语料库及置信区间计算方法。
原文摘要 · Abstract (English)
A speaker encoder used in multilingual voice cloning should treat the same speaker identically regardless of which script the audio was uttered in. Off-the-shelf encoders do not, and the failure is accent-conditional. On a 1043-pair Western-accented voice corpus across English, Hindi, Telugu, and Tamil, WavLM-base-plus-sv loses 0.082 absolute cosine similarity when the same voice changes script and ECAPA-TDNN loses 0.105. On a 1369-pair Indian-accented voice corpus, the gap shrinks to 0.006 (WavLM-SV) and 0.044 (ECAPA-TDNN). The leak is largest where it matters most for cross-script TTS: when a system projects a non-Indic-trained voice into Indic scripts. We present LASE (Language-Adversarial Speaker Encoder), a small projection head over frozen WavLM-base-plus trained with two losses: a supervised contrastive loss over voice identity, and a gradient-reversal cross-entropy against a 4-language classifier that pushes the embedding to be language-uninformative while remaining speaker-informative. Trained on 1118 quality-gated cross-script pairs synthesised from 8 commercial multilingual voices, LASE's residual gap is consistent with zero on both corpora (Delta = 0.013 Western, Delta = 0.026 Indian; both bootstrap 95% CIs include zero) and amplifies the cross-script-vs-floor margin 2.4-2.7x over both baselines. An ECAPA+GRL ablation shows the GRL objective improves either backbone but the WavLM choice contributes too. In synthetic multi-speaker diarisation, LASE matches ECAPA-TDNN on cross-script speaker recall (0.788 vs 0.789) with ~100x less training data. We release the r1 checkpoint, both corpora, and the bootstrap recipe.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。