用复数神经网络生成更自然的音频波形
Toward Complex-Valued Neural Networks for Waveform Generation
- 直接用复数运算处理频谱，捕捉相位结构
- 音质优于同类实数模型，训练快25%
- 适合音频生成、语音合成方向研究者
神经声码器近年显著提升了波形生成质量，其中基于iSTFT的方法因预测复数谱并直接通过逆短时傅里叶变换合成波形而受到关注，避免了高计算成本的上采样层。然而现有方法使用实数网络分别处理实部和虚部，难以捕捉复数谱的内在结构。本文提出ComVo，一种采用原生复数运算的声码器，其生成器与判别器均使用复数算术，支持在复数表示下进行对抗训练，获得结构化反馈。为引导相位变换，引入相位量化机制，对相位值离散化以正则化训练。此外，提出块矩阵计算方案，通过减少冗余操作提升训练效率。实验表明，ComVo在音质上优于对比的实数基线模型，且块矩阵方案使训练时间减少25%。音频样本与代码已公开于https://hs-oh-prml.github.io/ComVo/。
原文摘要 · Abstract (English)
Neural vocoders have recently advanced waveform generation, yielding natural and expressive audio. Among these approaches, iSTFT-based vocoders have recently gained attention. They predict a complex-valued spectrogram and then synthesize the waveform via iSTFT, thereby avoiding learned upsampling stages that can increase computational cost. However, current approaches use real-valued networks that process the real and imaginary parts independently. This separation limits their ability to capture the inherent structure of complex spectrograms. We present ComVo, a Complex-valued neural Vocoder whose generator and discriminator use native complex arithmetic. This enables an adversarial training framework that provides structured feedback in complex-valued representations. To guide phase transformations in a structured manner, we introduce phase quantization, which discretizes phase values and regularizes the training process. Finally, we propose a block-matrix computation scheme to improve training efficiency by reducing redundant operations. Experiments demonstrate that ComVo achieves higher synthesis quality than comparable real-valued baselines, and that its block-matrix scheme reduces training time by 25%. Audio samples and code are available at https://hs-oh-prml.github.io/ComVo/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。