用奇异值分解构建时序图频表示,提升语音增强中幅度与相位的对齐效果。
Time-Graph Frequency Representation with Singular Value Decomposition for Neural Speech Enhancement
- 基于SVD的图傅里叶变换生成实值时序图表示
- 相比STFT和EVD方法,在语音可懂度和感知质量上均更优
- 适合需要高保真语音重建的场景,如语音通信与听障辅助
单声道语音增强的时频域方法受益于深度学习的成功。近期研究聚焦于双流网络分别预测幅度掩码与相位,或将其耦合为笛卡尔坐标下的实虚数对。然而,大多数方法在双流框架下难以对齐幅度与相位(实虚数对)建模,导致性能受限。本文提出一种基于奇异值分解(GFT-SVD)定义的图傅里叶变换,生成用于神经语音增强的实值时序图表示。该实值表示能够有效对齐幅度与相位建模,避免了目标语音相位信息的恢复。实验表明,基于GFT-SVD的实值时序图表示在语音增强任务中表现优异。大量实验验证,GFT-SVD与DNN的结合优于GFT-EVD与幅度估计UNet的组合,也优于STFT与DNN,在客观可懂度和主观感知质量上均取得提升。代码已开源:https://github.com/Wangfighting0015/GFT_project。
原文摘要 · Abstract (English)
Time-frequency (T-F) domain methods for monaural speech enhancement have benefited from the success of deep learning. Recently, focus has been put on designing two-stream network models to predict amplitude mask and phase separately, or, coupling the amplitude and phase into Cartesian coordinates and constructing real and imaginary pairs. However, most methods suffer from the alignment modeling of amplitude and phase (real and imaginary pairs) in a two-stream network framework, which inevitably incurs performance restrictions. In this paper, we introduce a graph Fourier transform defined with the singular value decomposition (GFT-SVD), resulting in real-valued time-graph representation for neural speech enhancement. This real-valued representation-based GFT-SVD provides an ability to align the modeling of amplitude and phase, leading to avoiding recovering the target speech phase information. Our findings demonstrate the effects of real-valued time-graph representation based on GFT-SVD for neutral speech enhancement. The extensive speech enhancement experiments establish that the combination of GFT-SVD and DNN outperforms the combination of GFT with the eigenvector decomposition (GFT-EVD) and magnitude estimation UNet, and outperforms the short-time Fourier transform (STFT) and DNN, regarding objective intelligibility and perceptual quality. We release our source code at: https://github.com/Wangfighting0015/GFT\_project.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。