用周期性激活提升音频生成质量,解决分辨率低和模式崩溃问题。
DPN-GAN: Inducing Periodic Activations in Generative Adversarial Networks for High-Fidelity Audio Synthesis
- 引入基于核函数的周期性ReLU激活,增强音频模式捕捉能力。
- 采用可变形卷积实现多尺度生成,参数量达38.67M至124M。
- 在语音与音乐合成任务中均优于当前最佳模型,抗噪性强。
近年来,生成对抗网络(GAN)在音频生成方面取得显著进展。然而,这些模型通常依赖带宽受限的梅尔频谱图,限制了生成音频的分辨率,并导致条件生成时出现模式崩溃。为此,我们提出基于可变形周期网络的GAN(DPN-GAN),引入基于核函数的周期性ReLU激活函数,诱导音频生成中的周期性偏置,增强模型对复杂音频模式的捕捉与再现能力。该模型包含一个用于多分辨率生成的DPN模块,利用可变形卷积操作实现自适应感受野,提升合成音频的质量与保真度。同时,通过可变形卷积增强判别器,更精准区分真实与生成样本,进一步优化音频质量。我们训练了两个版本:小型(38.67M参数)与大型(124M参数)。在五个不同数据集上进行评估,涵盖语音合成与音乐生成任务,结果表明DPN-GAN在分布外数据与噪声数据下均表现优异,展现出更强鲁棒性与适应性。跨多种数据集训练后,其在标准评估指标上超越现有先进GAN架构,合成音频质量显著提升。
原文摘要 · Abstract (English)
In recent years, generative adversarial networks (GANs) have made significant progress in generating audio sequences. However, these models typically rely on bandwidth-limited mel-spectrograms, which constrain the resolution of generated audio sequences, and lead to mode collapse during conditional generation. To address this issue, we propose Deformable Periodic Network based GAN (DPN-GAN), a novel GAN architecture that incorporates a kernel-based periodic ReLU activation function to induce periodic bias in audio generation. This innovative approach enhances the model's ability to capture and reproduce intricate audio patterns. In particular, our proposed model features a DPN module for multi-resolution generation utilizing deformable convolution operations, allowing for adaptive receptive fields that improve the quality and fidelity of the synthetic audio. Additionally, we enhance the discriminator network using deformable convolution to better distinguish between real and generated samples, further refining the audio quality. We trained two versions of the model: DPN-GAN small (38.67M parameters) and DPN-GAN large (124M parameters). For evaluation, we use five different datasets, covering both speech synthesis and music generation tasks, to demonstrate the efficiency of the DPN-GAN. The experimental results demonstrate that DPN-GAN delivers superior performance on both out-of-distribution and noisy data, showcasing its robustness and adaptability. Trained across various datasets, DPN-GAN outperforms state-of-the-art GAN architectures on standard evaluation metrics, and exhibits increased robustness in synthesized audio.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。