轻量级音频编码器，高压采样低码率下仍保高音质。
MDCTCodec: A Lightweight MDCT-based Neural Audio Codec towards High Sampling Rate and Low Bitrate Scenarios
- 基于改进离散余弦变换的端到端编码，先转频谱再量化
- 48kHz下6kbps时ViSQOL达4.18，模型小且训练快
- 适合高采样率低码率场景，如远程通信或嵌入式设备
本文提出MDCTCodec，一种基于改进离散余弦变换（MDCT）的高效轻量级端到端神经音频编解码器。编码器将音频的MDCT频谱作为输入，编码为连续潜在码，并通过残差向量量化器（RVQ）离散化；解码器从量化后的潜在码重建MDCT频谱，并通过逆MDCT还原音频。训练阶段引入新型多分辨率MDCT判别器（MR-MDCTD），用于对抗性训练以区分自然与解码后的MDCT频谱。实验表明，在高采样率和低比特率场景下，相较于基线编解码器，MDCTCodec在音质、训练与生成效率以及模型规模方面均有显著提升。具体地，在公开数据集VCTK上，48kHz采样率、6kbps码率条件下，其ViSQOL评分为4.18。
原文摘要 · Abstract (English)
In this paper, we propose MDCTCodec, an efficient lightweight end-to-end neural audio codec based on the modified discrete cosine transform (MDCT). The encoder takes the MDCT spectrum of audio as input, encoding it into a continuous latent code which is then discretized by a residual vector quantizer (RVQ). Subsequently, the decoder decodes the MDCT spectrum from the quantized latent code and reconstructs audio via inverse MDCT. During the training phase, a novel multi-resolution MDCT-based discriminator (MR-MDCTD) is adopted to discriminate the natural or decoded MDCT spectrum for adversarial training. Experimental results confirm that, in scenarios with high sampling rates and low bitrates, the MDCTCodec exhibited high decoded audio quality, improved training and generation efficiency, and compact model size compared to baseline codecs. Specifically, the MDCTCodec achieved a ViSQOL score of 4.18 at a sampling rate of 48 kHz and a bitrate of 6 kbps on the public VCTK corpus.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。