提出新型神经语音压缩模型ECC，显著提升低码率下的音质与效率。
Benchmarking Neural Speech Compression from a Rate-Distortion Perspective
- 融合标量量化与自学习熵模型，统一编码流程
- 在两个测试集上平均降低BD-rate达39.9%~76.3%
- 引入熵跳过机制，无需额外传输掩码即可跳过可预测符号
基于学习的语音压缩已实现优异的低码率性能，但许多神经语音编解码器仍使用预设码率离散符号描述量化潜变量，或仅在符号生成后应用熵编码。此类设计将表示学习与概率建模分离，限制了对学习到的语音潜变量非均匀使用和时序依赖性的利用。本文从率失真角度对神经语音压缩进行基准测试，并进一步研究约束熵的编码方法。我们首先提出一个统一的学习型语音编码流程，对近期神经语音编解码器进行基准式分析，表明显式概率建模在学习型语音压缩中仍被低估。随后提出ECC（Entropy-Constrained Codec），结合标量量化与自学习熵模型，集成基于超先验的辅助信息、通道级上下文建模、潜变量残差预测及轻量时序建模，以在训练时估计潜变量似然用于率估计，在推理时进行算术编码。为提升低码率效率，ECC引入熵跳过机制，利用解码器可用的尺度估计跳过高度可预测的残差符号，无需传输额外跳过掩码。大量实验表明，ECC在两个广泛使用的测试集上相比传统和神经编解码器基线，实现了更优的率失真权衡，平均降低BD-rate达39.9%（ViSQOL）和76.3%（PESQ）。消融与诊断分析进一步验证了熵建模的有效性。
原文摘要 · Abstract (English)
Learning-based speech compression has achieved promising low-bitrate performance, but many neural speech codecs still describe quantized latents with preset-rate discrete symbols or apply entropy coding only after symbol generation. Such designs decouple representation learning from probability modeling, limiting their ability to exploit the non-uniform usage and temporal dependencies of learned speech latents. In this paper, we benchmark neural speech compression from a rate--distortion perspective and further investigate entropy-constrained coding for low-bitrate speech compression. We first formulate a unified learning-based speech coding pipeline and provide a benchmark-style analysis of recent neural speech codecs, showing that explicit probability modeling remains underexplored in learned speech compression. We then propose ECC, an Entropy-Constrained Codec that combines scalar quantization with a learned entropy model. ECC integrates hyperprior-based side information, channel-wise context modeling, latent residual prediction, and lightweight temporal modeling to estimate latent likelihoods for rate estimation during training and arithmetic coding during inference. To further improve low-bitrate efficiency, ECC introduces entropy skip, which omits highly predictable residual symbols using decoder-available scale estimates without transmitting additional skip masks. Extensive experiments show that ECC achieves a favorable low-bitrate rate--distortion trade-off over conventional and neural codec baselines, reducing BD-rate by 39.9% on ViSQOL and 76.3% on PESQ on average over two widely-used test sets. Ablation and diagnostic studies further validate the effectiveness of entropy modeling. Project Page: https://avery-xu.github.io/ECC-demo/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。