动态调整音频编码中码书数量，提升压缩效率。
Variable Bitrate Residual Vector Quantization for Audio Coding
- 根据音频复杂度动态调整每帧使用的码书数量。
- 在静音等简单音频上实现更低码率且无失真。
- 适用于追求高压缩比的音频编解码场景。
近期最先进的神经音频压缩模型逐步采用残差向量量化（RVQ）。尽管取得成功，这些模型每帧使用固定数量的码书，这在简单输入音频（如静音）场景下可能导致速率-失真权衡不佳。为解决此问题，我们提出可变比特率残差向量量化（VRVQ），通过按帧自适应调整码书数量，实现更高效的编码。此外，我们提出一种针对不可微掩码操作的梯度估计方法，将重要性图转换为二值重要性掩码，通过直通估计器改进模型训练。实验表明，所提训练框架优于基线方法，并在当前最先进编解码器上进一步提升性能。
原文摘要 · Abstract (English)
Recent state-of-the-art neural audio compression models have progressively adopted residual vector quantization (RVQ). Despite this success, these models employ a fixed number of codebooks per frame, which can be suboptimal in terms of rate-distortion tradeoff, particularly in scenarios with simple input audio, such as silence. To address this limitation, we propose variable bitrate RVQ (VRVQ) for audio codecs, which allows for more efficient coding by adapting the number of codebooks used per frame. Furthermore, we propose a gradient estimation method for the non-differentiable masking operation that transforms from the importance map to the binary importance mask, improving model training via a straight-through estimator. We demonstrate that the proposed training framework achieves superior results compared to the baseline method and shows further improvement when applied to the current state-of-the-art codec.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。