一个模型适配多种延迟需求，实时语音增强无需为不同场景重训练。
One Model, Many Latencies: Universal Speech Enhancement for Diverse Real-Time Applications
- 通过可配置前瞻帧灵活调节算法延迟，支持多场景部署。
- 采用并行卷积层与早退机制，实现计算延迟的动态控制。
- 两阶段训练策略提升通用模型性能，接近专用模型效果。
不同实时语音应用对延迟要求各异，通常需为每种场景单独训练增强模型。本文提出一种通用实时语音增强模型，可显式调控算法与计算延迟。算法延迟通过可配置的前瞻帧灵活调整；为避免不同填充配置导致的学习效率下降，引入对应不同前瞻设置的并行卷积层。计算延迟通过早退机制实现，支持在不同网络深度进行推理。为缩小通用模型与专用模型间的性能差距，提出共享到多解码器的两阶段训练策略。整体框架使单一模型可在无须重新训练的情况下，部署于多种延迟预算场景。模型权重已公开：https://huggingface.co/nvidia/Real-time_RE-USE
原文摘要 · Abstract (English)
Different real-time speech applications impose distinct latency budgets, often requiring separately trained enhancement models for each scenario. In this paper, we propose a one-for-all, real-time universal speech enhancement model that provides explicit control over both algorithmic and computational latency. Algorithmic latency is flexibly adjusted via configurable look-ahead frames. To avoid learning inefficiency caused by varying padding configurations, we introduce parallel convolutional layers corresponding to different look-ahead settings. Computational latency is controlled through an early-exit mechanism, enabling inference at different network depths. To narrow the performance gap between specialized and flexible models, we propose a two-stage training strategy with a shared-to-multiple decoder transition. Overall, the proposed framework enables a single model to be deployed across diverse latency budgets without retraining separate models. Model weights are available for download at: https://huggingface.co/nvidia/Real-time_RE-USE
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。