arXiv:2503.19597cs.SDeess.SP2025-03被引 3

用预训练编码器+离线量化,实现更简单高效的音频压缩。

QINCODEC: Neural Audio Compression with Implicit Neural Codebooks

  • 先预训练编码器,再离线使用任意量化器,无需反向传播设计
  • 在16kHz语音上达到4.39kbps码率,质量接近基准模型
  • 适合想快速实验新量化方法的研究者或工业应用

神经音频编解码器在音频生成模型发展中起关键作用。现有先进编解码器依赖端到端训练的自编码器与量化瓶颈,但受限于需定义量化器梯度传播方式及在线更新参数。本文提出新策略:对预训练自编码器的潜在表示进行离线量化,并可选地微调解码器以缓解量化带来的失真。该方法使任意现成量化器可用,尤其适用于具有隐式神经码本的先进可训练量化器(如QINCO2)。我们提出的编解码器QINCODEC在性能上媲美基线模型,同时训练更简单。该方法提供通用框架,分摊自编码器预训练成本,支持更灵活的编解码器设计。

原文摘要 · Abstract (English)

Neural audio codecs, neural networks which compress a waveform into discrete tokens, play a crucial role in the recent development of audio generative models. State-of-the-art codecs rely on the end-to-end training of an autoencoder and a quantization bottleneck. However, this approach restricts the choice of the quantization methods as it requires to define how gradients propagate through the quantizer and how to update the quantization parameters online. In this work, we revisit the common practice of joint training and propose to quantize the latent representations of a pre-trained autoencoder offline, followed by an optional finetuning of the decoder to mitigate degradation from quantization. This strategy allows to consider any off-the-shelf quantizer, especially state-of-the-art trainable quantizers with implicit neural codebooks such as QINCO2. We demonstrate that with the latter, our proposed codec termed QINCODEC, is competitive with baseline codecs while being notably simpler to train. Finally, our approach provides a general framework that amortizes the cost of autoencoder pretraining, and enables more flexible codec design.

音频压缩神经码本离线量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。