无需解耦,直接控制音高和时长,实现灵活语音转换
Fast-VGAN: Lightweight Voice Conversion with Explicit Control of F0 and Duration Parameters
- 通过显式条件控制音高、音素序列和说话人特征生成声谱图
- 在语音转换与情感表达任务中保持高可懂度与说话人相似性
- 支持推理时自由调节音高轮廓,适合语音编辑与个性化合成
精确控制语音特征(如音高、时长、语速)仍是语音转换领域的关键挑战。本文提出一种基于卷积神经网络的方法,直接对基频(F0)、音素序列、强度和说话人身份进行显式建模,生成梅尔频谱图,并通过通用神经声码器还原为波形。推理时可自由调整音高轮廓、音素序列和说话人嵌入,实现直观可控的语音变换。在说话人转换与情感表达任务上,结合感知与客观指标评估,结果表明该方法具备显著灵活性,同时保持高可懂度与说话人相似性。
原文摘要 · Abstract (English)
Precise control over speech characteristics, such as pitch, duration, and speech rate, remains a significant challenge in the field of voice conversion. The ability to manipulate parameters like pitch and syllable rate is an important element for effective identity conversion, but can also be used independently for voice transformation, achieving goals that were historically addressed by vocoder-based methods. In this work, we explore a convolutional neural network-based approach that aims to provide means for modifying fundamental frequency (F0), phoneme sequences, intensity, and speaker identity. Rather than relying on disentanglement techniques, our model is explicitly conditioned on these factors to generate mel spectrograms, which are then converted into waveforms using a universal neural vocoder. Accordingly, during inference, F0 contours, phoneme sequences, and speaker embeddings can be freely adjusted, allowing for intuitively controlled voice transformations. We evaluate our approach on speaker conversion and expressive speech tasks using both perceptual and objective metrics. The results suggest that the proposed method offers substantial flexibility, while maintaining high intelligibility and speaker similarity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。