用离散表示提升语音降噪，兼顾效果与低延迟。
LL-SDR: Low-Latency Speech enhancement through Discrete Representations
- 通过离散化分离语音与噪声分布，提升降噪清晰度。
- 单次推理仅需40G MACs，GPU上实时率低至0.01。
- 适合对延迟敏感的实时语音处理场景。
许多语音增强（SE）方法依赖于连续表示。最近，离散音频标记被用于实现自回归生成的语音增强。然而，离散化是否能持续提升性能仍不明确。本文提出LL-SDR，一种基于标记的语音增强框架，显式利用离散化以更好分离语音与噪声。第一项贡献是方差有序残差向量量化器（VO-RVQ），在标记化过程中解耦语音与噪声分布。第二项是潜在空间判别器，使增强嵌入更接近语义嵌入。实验表明，LL-SDR优于连续基线，并达到自回归标记方法的性能。尽管增强效果强，其仍轻量高效，单次前向传播仅需40G MACs，10秒16kHz语音段在GPU上实时率（RTF）为0.01，在CPU上为0.24。演示与源码见项目网站。
原文摘要 · Abstract (English)
Many speech enhancement (SE) methods rely on continuous representations. Recently, discrete audio tokens have been explored to enable autoregressive generation for SE. However, it remains unclear whether discretization itself consistently improves SE performance. In this paper, we introduce LL-SDR, a token-based speech enhancement framework that explicitly leverages discretization to better separate speech and noise. Our first contribution is a Variance-Ordered Residual Vector Quantizer (VO-RVQ), designed to disentangle speech and noise distributions during tokenization. Second, we propose a latent-space discriminator to better align enhanced embeddings with semantic embeddings. Experiments show that LL-SDR outperforms continuous baselines and matches the performance of autoregressive token-based approaches. Despite its strong enhancement performance, LL-SDR remains lightweight and efficient, requiring only 40G MACs for a single forward pass on a 10-second 16 kHz speech segment and achieving low-latency inference with an RTF of 0.01 on GPU and 0.24 on CPU. Demos and source code are available at our project websites.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。