通过模拟量化扰动提升语音编码器抗噪能力，无需噪声数据。
Enhancing Noise Robustness for Neural Speech Codecs through Resource-Efficient Progressive Quantization Perturbation Simulation
- 在量化层直接模拟噪声扰动，用概率采样替代固定匹配。
- 在15dB信噪比下，Encodec的语音质量评分从3.475提升至3.586。
- 仅用纯净语音训练，适合实际部署中噪声环境复杂场景。
噪声鲁棒性是神经语音编码器在真实声学环境中部署的关键挑战。我们发现，即使轻微的输入噪声扰动也会导致量化码字意外偏移，从而降低重建语音质量。为此，提出一种资源高效的新训练策略，在量化层级直接模拟此类扰动。方法包含两个核心机制：(1) 使用距离加权的概率性top-K采样，替代残差向量量化（RVQ）中的确定性最近邻选择；(2) 采用渐进式训练方案，从最后一个量化器开始逐步引入可控扰动。关键优势在于，模型仅在纯净语音上训练，无需任何成对的噪声-纯净语音数据。在两个先进语音编码器Encodec和WavTokenizer上的实验表明，该策略显著提升了噪声条件下的鲁棒性——例如，在15 dB SNR下，Encodec的UTMOS从3.475提升至3.586——同时优化了纯净语音的编码质量。
原文摘要 · Abstract (English)
Noise robustness remains a critical challenge for deploying neural speech codecs in real-world acoustic scenarios where background noise is often inevitable. A key observation we make is that even slight input noise perturbations can cause unintended shifts in quantized codewords, thereby degrading the quality of reconstructed speech. Motivated by this finding, we propose a novel and resource-efficient training strategy to enhance the noise robustness of speech codecs by simulating such perturbations directly at the quantization level. Our approach introduces two core mechanisms: (1) a distance-weighted probabilistic top-K sampling strategy that replaces the conventional deterministic nearest-neighbor selection in residual vector quantization (RVQ); and (2) a progressive training scheme that introduces perturbations from the last to the first quantizer in a controlled manner. Crucially, our method is trained exclusively on clean speech, eliminating the need for any paired noisy-clean data. Experiments on two advanced neural speech codecs, Encodec and WavTokenizer, demonstrate that the proposed strategy substantially improves robustness under noisy conditions-for example, boosting UTMOS from 3.475 to 3.586 at 15 dB SNR on Encodec-while also enhancing coding quality for clean speech.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。