用伪配对数据提升单次语音转换效果，解决训练与推理不一致问题。
PseudoVC: Improving One-shot Voice Conversion with Pseudo Paired Data
- 通过伪转换生成扰动语音，缓解内容编码器输入不一致问题。
- 用同说话人另一段语音替代原输入，解决说话人编码器匹配难题。
- 在公开模型中表现更优，适合语音转换研究者使用。
由于单次语音转换（One-shot VC）任务中并行训练数据稀缺，通常依赖各类语音转换系统进行波形重建。典型的一次性语音转换系统包含内容编码器和说话人编码器，但训练与推理时存在两类输入不匹配：内容编码器输入不一致，以及说话人编码器输入不一致。为此，本文提出一种新型训练方法 PseudoVC。首先，引入名为“伪转换”的信息扰动方法，利用预训练的语音转换模型将源语音转换为扰动语音，作为内容编码器的训练输入；其次，提出“说话人采样”方法，在训练时用同一说话人的另一段语音替换原输入，以解决说话人编码器的输入不匹配问题。实验结果表明，所提出的伪转换方法优于以往的信息扰动方法，整体的 PseudoVC 方法超越了现有公开的语音转换模型。音频示例已提供。
原文摘要 · Abstract (English)
As parallel training data is scarce for one-shot voice conversion (VC) tasks, waveform reconstruction is typically performed by various VC systems. A typical one-shot VC system comprises a content encoder and a speaker encoder. However, two types of mismatches arise: one for the inputs to the content encoder during training and inference, and another for the inputs to the speaker encoder. To address these mismatches, we propose a novel VC training method called \textit{PseudoVC} in this paper. First, we introduce an innovative information perturbation approach named \textit{Pseudo Conversion} to tackle the first mismatch problem. This approach leverages pretrained VC models to convert the source utterance into a perturbed utterance, which is fed into the content encoder during training. Second, we propose an approach termed \textit{Speaker Sampling} to resolve the second mismatch problem, which will substitute the input to the speaker encoder by another utterance from the same speaker during training. Experimental results demonstrate that our proposed \textit{Pseudo Conversion} outperforms previous information perturbation methods, and the overall \textit{PseudoVC} method surpasses publicly available VC models. Audio examples are available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。