PACodec用并行分量量化，30%更低码率仍保高质量。
PACodec: A Low-bitrate Neural Speech Codec with Parallel Additive Vector Quantization
- 采用并行加性向量量化，取代传统串行依赖结构。
- 在同等音质下比特率降低30%，仅小幅增加模型复杂度。
- 各量化器解耦语音内容、音色等特征，适合语音转换等任务。
本文提出PACodec，一种基于并行加性向量量化（PAVQ）的新型低码率神经语音编解码器。与主流的残差向量量化（RVQ）中量化器存在序列依赖不同，PAVQ通过并行聚合多个独立量化器的结果来优化码率使用。具体地，PAVQ采用“全局-局部-全局”（GLG）设计：全局编码特征由多个独立的向量量化器（VQs）并行处理，每个关注表示的一个局部成分，其输出通过相加得到最终的全局量化结果用于解码。实验表明，由于每个VQ仅聚焦局部信息，PACodec支持更小的码本，相较于基线在相同解码质量下降低30%码率，且模型复杂度仅轻微增加。进一步分析显示，得益于PAVQ的GLG框架，PACodec具备解耦友好性，各独立VQ分别捕捉语音的内容、音色和声学细节等不同方面，为语音转换等下游任务提供了潜在应用价值。
原文摘要 · Abstract (English)
This paper proposes PACodec, a novel low-bitrate neural speech codec based on parallel additive vector quantization (PAVQ). Unlike the mainstream residual vector quantization (RVQ) used in most neural speech codecs, where vector quantizers (VQs) are sequentially dependent, the PAVQ strategy adopted in PACodec aggregates parallel quantization results to optimize bitrate usage. Specifically, the PAVQ adopts a "global-local-global" (GLG) design: the global encoded features are quantized in parallel by multiple independent VQs, each attending to a local component of the representation, and their outputs are aggregated through addition to yield the final global quantization result for decoding. Experimental results show that PACodec, as each VQ focuses only on local information, supports smaller codebooks and reduces bitrate by 30% compared with baselines at the same decoding quality, with only minor model complexity. Further analysis shows that, owing to the GLG framework of PAVQ, the proposed PACodec is disentanglement-friendly, and each independent VQ captures different aspects of speech, e.g., content, timbre, and acoustic details, suggesting potential for application to downstream tasks such as voice conversion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。