根据语音感知敏感度，动态分配比特率，显著降低神经语音编码的码率。
VoCodec: A Low-bitrate Streamable Neural Speech Codec with Voicing-driven Quantization
- 按音段类型动态分配比特率，语音帧用高精度量化，非语音帧用低精度。
- 在1.1 kbps下仍优于基线模型，比均匀量化节省约27%码率。
- 适合低延迟流媒体传输，尤其对带宽敏感的语音通信场景。
神经语音编码器是语音传输与存储的关键技术，但多数方法对所有帧采用统一量化，不区分内容，造成比特浪费。本文提出VoCodec，一种支持流式传输的低码率神经语音编码器，基于发声特性驱动的量化策略：根据感知敏感度，为有声帧分配更高码率，无声帧分配更低码率。该模型在全因果的编码-量化-解码框架中嵌入发声检测器，对有声帧使用残差标量向量量化，对无声帧采用简单标量量化。在16 kHz采样率的LibriTTS数据集上的实验表明，即使在1.1 kbps的极低码率下，VoCodec仍优于现有基线模型；进一步实验验证，引入发声驱动量化可使码率相较均匀量化降低约27%。
原文摘要 · Abstract (English)
Neural speech codecs are key to speech transmission and storage, but most use uniform quantization across frames, allocating the same bitrate regardless of content and wasting bits. We propose VoCodec, a low-bitrate streamable neural speech codec with voicing-driven quantization that assigns higher bitrate to voiced frames and lower bitrate to unvoiced frames according to perceptual sensitivity. VoCodec embeds a voicing detector in a fully causal encoder-quantizer-decoder neural coding framework, using residual scalar-vector quantization for voiced frames and simple scalar quantization for unvoiced ones. Experiments show that on the LibriTTS dataset at a 16 kHz sampling rate, VoCodec outperforms baseline neural speech codecs even at a bitrate as low as 1.1 kbps. Our further experiments also confirm that introducing voicing-driven quantization can effectively reduce the bitrate by approximately 27% compared with uniform quantization strategy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。