无需成对数据，用KNN检索实现跨语言语音转换。
From A to B to A: Palindromic Zero-Shot Voice Conversion with Non-Parallel Data
- 用WavLM特征检索匹配音段，构造合成训练对。
- 仅用英语数据训练，跨语言转换仍自然且保声线。
- 适合无平行语料的多语言语音转换场景。
我们提出一种语音转换框架，利用WavLM表征的K近邻检索，对非平行源语音与目标语音进行对齐，构建用于监督学习的合成训练样本。检索到的音段作为合成输入，真实目标音频作为真实输出，形成合成到真实的训练范式，天然支持多语言数据而无需并行语料或显式对齐。为确保目标说话人身份一致，引入基于预训练说话人验证模型的说话人损失。在多种语言上的实验表明，该方法在仅使用英语数据训练的情况下，仍能实现高自然度和强说话人相似性，优于现有竞争性基线。样例可访问：https://palindromic-vc.github.io。
原文摘要 · Abstract (English)
We present a voice conversion (VC) framework that utilizes K-Nearest Neighbors (KNN) retrieval over WavLM representations to align non-parallel source and target speech, constructing synthetic training pairs for supervised learning. The retrieved segments serve as synthetic inputs, while real target audio provides ground-truth outputs, forming a synthetic-to-real training paradigm that naturally supports multilingual data without requiring parallel corpora or explicit alignment. To ensure consistent target-speaker identity, we incorporate a speaker loss derived from a pretrained speaker verification model. Experiments across multiple languages demonstrate that the proposed approach achieves high naturalness and strong speaker similarity, outperforming competitive VC baselines, despite being trained exclusively on English data. Samples can be accessed at: https://palindromic-vc.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。