用预训练语音模型将近场语音转为远场语音,提升远场说话人验证性能
Adaptive Data Augmentation with NaturalSpeech3 for Far-field Speaker Verification
- 利用NaturalSpeech3拆解语音特征,混合远场环境与近场声纹生成伪远场语音
- 在FFSVC数据集上显著优于随机加噪、混响等传统增强方法
- 适合远场说话人验证、跨数据集验证场景,尤其缺远场标注数据时
远场语音说话人验证系统面临标注说话人数据稀缺的挑战。尽管使用大规模近场语音进行数据增强是常见策略,但近场与远场声学环境差异严重限制了增强效果。本文提出一种自适应语音增强方法,基于预训练的NaturalSpeech3文本到语音模型,通过引入远场环境噪声,将近场语音转换为远场语音。具体地,采用NaturalSpeech3中的FACodec将语音波形分解为内容、语调、说话人和残差(声学细节)嵌入,并重构语音。本方法将远场语音的语调、内容和残差嵌入与近场语音的说话人嵌入结合,生成保留近场说话人身份但具备远场声学环境的伪远场语音。该方法不仅有效扩充远场说话人验证训练数据,还可用于注册与测试语音的跨数据集增强。在FFSVC数据集上的实验表明,该方法显著优于随机加噪、混响等传统及先进增强策略。
原文摘要 · Abstract (English)
The scarcity of speaker-annotated far-field speech presents a significant challenge in developing high-performance far-field speaker verification (SV) systems. While data augmentation using large-scale near-field speech has been a common strategy to address this limitation, the mismatch in acoustic environments between near-field and far-field speech significantly hinders the improvement of far-field SV effectiveness. In this paper, we propose an adaptive speech augmentation approach leveraging NaturalSpeech3, a pre-trained foundation text-to-speech (TTS) model, to convert near-field speech into far-field speech by incorporating far-field acoustic ambient noise for data augmentation. Specifically, we utilize FACodec from NaturalSpeech3 to decompose the speech waveform into distinct embedding subspaces-content, prosody, speaker, and residual (acoustic details) embeddings-and reconstruct the speech waveform from these disentangled representations. In our method, the prosody, content, and residual embeddings of far-field speech are combined with speaker embeddings from near-field speech to generate augmented pseudo far-field speech that maintains the speaker identity from the out-domain near-field speech while preserving the acoustic environment of the in-domain far-field speech. This approach not only serves as an effective strategy for augmenting training data for far-field speaker verification but also extends to cross-data augmentation for enrollment and test speech in evaluation trials.Experimental results on FFSVC demonstrate that the adaptive data augmentation method significantly outperforms traditional approaches, such as random noise addition and reverberation, as well as other competitive data augmentation strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。