用扩散模型优化语音识别中的说话人嵌入，提升环境变化下的识别准确率。
SEED: Speaker Embedding Enhancement Diffusion Model
- 基于扩散模型对说话人嵌入进行去噪重构，无需标签和流程修改。
- 在模拟环境不匹配场景下，识别准确率最高提升19.6%。
- 适用于现有系统无缝升级，尤其适合复杂噪声环境应用。
实际部署中，说话人识别系统常因环境差异导致性能下降。本文提出一种基于扩散模型的方法，对预训练模型提取的说话人嵌入进行优化生成。训练阶段，通过扩散模型前向过程分别向干净与嘈杂语音提取的嵌入添加高斯噪声；反向过程中重建为纯净嵌入。推理时，所有嵌入均通过扩散过程重生成。本方法无需说话人标签，且不改变原有识别流程。在模拟环境不匹配的评估集上，相比基线模型，识别准确率最高提升19.6%，同时保持常规场景性能。代码已开源：https://github.com/kaistmm/seed-pytorch。
原文摘要 · Abstract (English)
A primary challenge when deploying speaker recognition systems in real-world applications is performance degradation caused by environmental mismatch. We propose a diffusion-based method that takes speaker embeddings extracted from a pre-trained speaker recognition model and generates refined embeddings. For training, our approach progressively adds Gaussian noise to both clean and noisy speaker embeddings extracted from clean and noisy speech, respectively, via forward process of a diffusion model, and then reconstructs them to clean embeddings in the reverse process. While inferencing, all embeddings are regenerated via diffusion process. Our method needs neither speaker label nor any modification to the existing speaker recognition pipeline. Experiments on evaluation sets simulating environment mismatch scenarios show that our method can improve recognition accuracy by up to 19.6% over baseline models while retaining performance on conventional scenarios. We publish our code here https://github.com/kaistmm/seed-pytorch
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。