用自注意力机制提升语音转换的全局语义捕捉能力
VCTR: A Transformer-Based Model for Non-parallel Voice Conversion
- 采用混合感知块与双剪枝自注意力结构，增强长距离依赖建模
- 在非配对数据下实现更自然的语音转换效果，优于传统CNN方法
- 适合语音合成、语音克隆等需跨域声线转换的研究者
非配对语音转换旨在不依赖成对训练数据的情况下，将源域语音转换为目标域语音。尽管循环一致生成对抗网络（CycleGAN）和变分自编码器（VAE）已被用于该任务，但其训练困难且效果不佳。后续提出的对比语音转换（CVC）采用基于对比学习的方法改善问题，但仍使用基于卷积神经网络（CNN）的生成器，虽能捕捉局部语义，却难以建模长程依赖，影响全局语义表达。本文提出VCTR，一种基于Transformer的高效非配对语音转换方法，结合混合感知块（HPB）与双剪枝自注意力（DPSA），并引入基于对比学习的对抗框架。实验表明，该方法在无配对数据条件下显著提升语音转换质量。
原文摘要 · Abstract (English)
Non-parallel voice conversion aims to convert voice from a source domain to a target domain without paired training data. Cycle-Consistent Generative Adversarial Networks (CycleGAN) and Variational Autoencoders (VAE) have been used for this task, but these models suffer from difficult training and unsatisfactory results. Later, Contrastive Voice Conversion (CVC) was introduced, utilizing a contrastive learning-based approach to address these issues. However, these methods use CNN-based generators, which can capture local semantics but lacks the ability to capture long-range dependencies necessary for global semantics. In this paper, we propose VCTR, an efficient method for non-parallel voice conversion that leverages the Hybrid Perception Block (HPB) and Dual Pruned Self-Attention (DPSA) along with a contrastive learning-based adversarial approach. The code can be found in https://github.com/Maharnab-Saikia/VCTR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。