用预测伪向量降低语音编码比特率，0.5kbps实现2kbps效果。
An Ultra-Low-Bitrate Neural Speech Codec with Plain-to-Pseudo Synergistic Vector Quantization
- 用真实向量和预测伪向量协同压缩，减少传输数据量。
- 在0.5kbps下达到2kbps的语音质量，比传统方法效率高4倍。
- 适合极低带宽场景，如远程通信或物联网语音传输。
大多数神经语音编码器采用残差向量量化（RVQ），后续量化层贡献小却占用相同比特率，造成效率低下。本文提出P2PSynCodec，一种超低比特率神经语音编码器，采用从纯量到伪量协同的向量量化器（P2PSVQ）。P2PSVQ包含一个真实向量量化器（plain VQ）和多个由神经网络预测生成的伪向量量化器（pseudo VQs）。真实VQ生成基础码本项，伪VQ通过预测生成辅助码本项且不消耗传输比特率。解码时结合真实码本项与预测的伪码本项重建语音，显著降低比特率。实验表明，P2PSynCodec在0.5 kbps下可实现与2.0 kbps竞争性编码器相当的语音重建质量，验证了其在超低比特率语音编码中的高效性。
原文摘要 · Abstract (English)
Most neural speech codecs use residual vector quantization (RVQ), in which later VQs contribute less but consume the same bitrate, leading to inefficiency. We propose P2PSynCodec, an ultra-low-bitrate neural speech codec with a plain-to-pseudo synergistic vector quantizer (P2PSVQ). P2PSVQ consists of one plain VQ and multiple pseudo VQs. The plain VQ produces basic tokens by quantization, while the pseudo VQs generate auxiliary tokens by neural prediction and incur zero transmitted bitrate. Thus, speech is decoded from the plain-VQ tokens together with predicted pseudo-VQ tokens, greatly reducing bitrate. Experiments show that P2PSynCodec achieves speech reconstruction quality comparable to competing codecs at 2.0 kbps while operating at only 0.5 kbps, demonstrating high efficiency for ultra-low-bitrate speech coding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。