用神经前向滤波建模混响,分离说话人音频同时保留混响特征。
Neural Forward Filtering for Speaker-Image Separation
- 构建双神经网络结构,中间插入神经前向滤波模块
- 在SMS-WSJ数据集上语音分离效果优于传统端到端方法
- 适合需要保留原始混响特性的语音分离场景
针对混响环境下单通道多人语音图像分离问题,目标是在分离出各说话人语音的同时保留其原始混响特性。传统端到端深度神经网络直接预测混响语音,但未显式利用物理约束:混响语音可通过直达信号与线性滤波器卷积生成。为此,提出CxNet——一种包含神经前向滤波模块的双神经网络系统。第一阶段网络联合预测直达信号与混响语音;基于直达信号估计,神经前向滤波模块计算线性滤波器,并将其与直达信号卷积生成新的混响语音估计,作为判别特征输入第二阶段网络以优化混响语音估计。通过显式建模线性滤波器,CxNet利用直达信号与混响语音间的物理关系,捕捉混响尾部关键信息。在SMS-WSJ数据集上的实验验证了该方法的有效性。
原文摘要 · Abstract (English)
We address monaural multi-speaker-image separation in reverberant conditions, aiming at separating mixed speakers but preserving the reverberation of each speaker. A straightforward approach for this task is to directly train end-to-end DNN systems to predict the reverberant speech of each speaker based on the input mixture. Although effective, this approach does not explicitly exploit the physical constraint that reverberant speech can be reproduced by convolving the direct-path signal with a linear filter. To address this, we propose CxNet, a two-DNN system with a neural forward filtering module in between. The first DNN is trained to jointly predict the direct-path signal and reverberant speech. Based on the direct-path estimate, the neural forward filtering module estimates the linear filter, and the estimated filter is then convolved with the direct-path estimate to obtain another estimate of reverberant speech, which is utilized as a discriminative feature to help the second DNN better estimate the reverberant speech. By explicitly modeling the linear filter, CxNet could leverage the physical constraint between the direct-path signal and reverberant speech to capture crucial information about reverberation tails. Evaluation results on the SMS-WSJ dataset show the effectiveness of the proposed algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。