通过分层注入条件嵌入，提升语音增强对复杂噪声的适应能力
SLICE: Speech Enhancement via Layer-wise Injection of Conditioning Embeddings
- 将退化条件嵌入注入时间步嵌入，贯穿所有残差块
- 在复合退化场景下，性能优于仅输入层注入的方法
- 适用于真实录音，无需修改模型结构
现实语音常同时受到多种退化影响，包括加性噪声、混响和非线性失真。基于扩散的增强方法在单一退化上表现良好，但在复合退化下效果下降。以往的噪声感知方法仅在输入层注入条件，可能导致性能低于无条件模型。为此，我们提出从预训练编码器（含噪声类型、混响、失真多任务头）提取退化条件，并注入到时间步嵌入中，使其贯穿所有残差块，无需改变模型架构。在仅变化注入方式的受控实验中，输入层注入在复合退化下表现劣于无编码器的情况，而分层注入取得最佳效果。该方法也适用于多样化的真实录音场景。
原文摘要 · Abstract (English)
Real-world speech is often corrupted by multiple degradations simultaneously, including additive noise, reverberation, and nonlinear distortion. Diffusion-based enhancement methods perform well on single degradations but struggle with compound corruptions. Prior noise-aware approaches inject conditioning at the input layer only, which can degrade performance below that of an unconditioned model. To address this, we propose injecting degradation conditioning, derived from a pretrained encoder with multi-task heads for noise type, reverberation, and distortion, into the timestep embedding so that it propagates through all residual blocks without architectural changes. In controlled experiments where only the injection method varies, input-level conditioning performs worse than no encoder at all on compound degradations, while layer-wise injection achieves the best results. The method also generalizes to diverse real-world recordings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。