多尺度编码提升低比特率语音压缩效果
Optimizing Neural Speech Codec for Low-Bitrate Compression via Multi-Scale Encoding
- 按不同时间尺度分层编码，匹配语音特征的信息密度差异
- 在低比特率下显著提升压缩性能，减少冗余信息编码
- 适合需要高效语音压缩的通信与存储场景
神经语音编解码器通过将语音转换为离散标记表示，实现了高质量语音与音频的压缩。现有方法大多采用残差向量量化（RVQ）以统一时间尺度对语音进行多层离散编码，但忽略了不同语音特征间的信息密度差异，导致稀疏信息被冗余编码，限制了其在低比特率下的表现。本文提出MsCodec，一种新型多尺度神经语音编解码器，将语音编码为对应不同时间尺度的多层离散码，促使模型根据信息密度差异分离语音特征，从而提升压缩性能。此外，引入互信息损失以增强各层间语音码的多样性。实验表明，该方法在低比特率下显著改善了编解码器性能。
原文摘要 · Abstract (English)
Neural speech codecs have demonstrated their ability to compress high-quality speech and audio by converting them into discrete token representations. Most existing methods utilize Residual Vector Quantization (RVQ) to encode speech into multiple layers of discrete codes with uniform time scales. However, this strategy overlooks the differences in information density across various speech features, leading to redundant encoding of sparse information, which limits the performance of these methods at low bitrate. This paper proposes MsCodec, a novel multi-scale neural speech codec that encodes speech into multiple layers of discrete codes, each corresponding to a different time scale. This encourages the model to decouple speech features according to their diverse information densities, consequently enhancing the performance of speech compression. Furthermore, we incorporate mutual information loss to augment the diversity among speech codes across different layers. Experimental results indicate that our proposed method significantly improves codec performance at low bitrate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。