用强化学习优化300bps语音编码，提升可懂度
Optimising Neural Speech Codecs for 300bps Communication using Reinforcement Learning
- 将量化设计为随机策略，用纠错率奖励训练编码器
- 300bps下词错误率降至3.55%，相对降低23.5%
- 适用于卫星、水下等低带宽场景，支持实时流传输
在卫星和水下通信等带宽受限场景中，语音常需以极低码率传输，此时可懂度是首要目标。传统基于声学重建损失的编解码器在极端压缩下会分配比特于感知细节，导致词错误率（WER）显著上升。本文提出ClariCodec，一种运行在300 bps的神经语音编解码器，将量化重构为随机策略，实现基于强化学习（RL）的可懂度优化。具体地，编码器使用以WER为导向的奖励进行微调，而声学重建流程保持冻结。即使无强化学习，ClariCodec在LibriSpeech test-clean集上已达4.64% WER，已优于部分高码率编解码器。进一步强化学习微调使WER降至3.55%，相对降低23.5%，同时保持良好听感质量。此外，我们将其适配为流式配置，在理论延迟374 ms下仍能实现测试集4.53% WER，证明其在实时约束下的有效性。
原文摘要 · Abstract (English)
In bandwidth-constrained communication such as satellite and underwater channels, speech must often be transmitted at ultra-low bitrates where intelligibility is the primary objective. At such extreme compression levels, codecs trained with acoustic reconstruction losses tend to allocate bits to perceptual detail, leading to substantial degradation in word error rate (WER). This paper proposes ClariCodec, a neural speech codec operating at 300 bits per second (bps) that reformulates quantisation as a stochastic policy, enabling reinforcement learning (RL)-based optimisation of intelligibility. Specifically, the encoder is fine-tuned using WER-driven rewards while the acoustic reconstruction pipeline remains frozen. Even without RL, ClariCodec achieves 4.64% WER on the LibriSpeech test-clean set at 300 bps, already competitive with codecs operating at higher bitrates. Further RL fine-tuning reduces WER to 3.55% on test-clean, corresponding to a 23.5% relative reduction while preserving perceptual quality. In addition, we adapt ClariCodec to a streaming configuration and show that the proposed RL-based optimisation remains effective under streaming constraints, achieving 4.53% WER on test-clean with a theoretical latency of 374 ms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。