纯Transformer架构+三元组训练，实现单样本语音转换
Pureformer-VC: Non-parallel One-Shot Voice Conversion with Pure Transformer Blocks and Triplet Discriminative Training
- 用Conformer和Zipformer构建解耦编码器与风格迁移解码器
- 在单样本场景下主观评分相当，客观指标优于现有方法
- 适合语音克隆、个性化语音合成等实际应用
单样本语音转换（One-shot VC）旨在仅用一个目标说话人语音样本，将任意源语音的音色转换为目标说话人。现有基于风格迁移的方法依赖语音表征解耦，但存在各语音成分编码不精确、重组困难的问题。为此，我们提出Pureformer-VC，采用Conformer块构建解耦编码器，用Zipformer块构建风格迁移解码器作为生成器，并在解码器中引入有效的StyleFormer块，将说话人特征有效融入生成语音。模型使用生成式变分自编码器（VAE）损失进行编码组件建模，结合三元组损失进行无监督判别训练。通过在Zipformer共享权重中应用StyleFormer方法实现风格迁移。实验表明，该模型在单样本语音转换场景下达到可比的主观评分，且在客观指标上优于现有方法。
原文摘要 · Abstract (English)
One-shot voice conversion(VC) aims to change the timbre of any source speech to match that of the target speaker with only one speech sample. Existing style transfer-based VC methods relied on speech representation disentanglement and suffered from accurately and independently encoding each speech component and recomposing back to converted speech effectively. To tackle this, we proposed Pureformer-VC, which utilizes Conformer blocks to build a disentangled encoder, and Zipformer blocks to build a style transfer decoder as the generator. In the decoder, we used effective styleformer blocks to integrate speaker characteristics effectively into the generated speech. The models used the generative VAE loss for encoding components and triplet loss for unsupervised discriminative training. We applied the styleformer method to Zipformer's shared weights for style transfer. The experimental results show that the proposed model achieves comparable subjective scores and exhibits improvements in objective metrics compared to existing methods in a one-shot voice conversion scenario.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。