arXiv:2502.14224eess.AScs.SD2025-02被引 4

让卷积神经网络动态调整卷积核,提升语音增强效果

Adaptive Convolution for CNN-based Speech Enhancement Models

  • 通过动态生成每帧的卷积核,自适应捕捉语音频谱特征
  • 在轻量模型上性能显著提升,计算开销几乎不变
  • 适合对实时性要求高的语音增强场景

基于深度学习的语音增强方法显著提升了语音质量和可懂度。卷积神经网络(CNN)已被证明是高性能模型的关键组件。本文提出自适应卷积,一种高效且通用的卷积模块,可增强模型对语音信号的自适应表示能力。该模块执行帧级因果动态卷积,通过组合多个并行候选卷积核,为每帧生成时变卷积核。我们设计了一种轻量级注意力机制,利用当前及历史信息为每个候选核分配自适应权重,使卷积操作能够适应帧级语音频谱特征,实现更高效的特征提取与重建。我们将自适应卷积集成到多种CNN模型中,验证其通用性。实验结果表明,该模块显著提升性能,同时计算复杂度几乎无增加,尤其在轻量模型上表现突出。此外,我们通过直观分析发现卷积核选择与信号特征存在强相关性。进一步提出自适应卷积递归网络(AdaptCRN),结合自适应卷积与高效编码器-解码器结构,成为超轻量模型,在相同或更高计算成本下表现更优。

原文摘要 · Abstract (English)

Deep learning-based speech enhancement methods have significantly improved speech quality and intelligibility. Convolutional neural networks (CNNs) have been proven to be essential components of many high-performance models. In this paper, we introduce adaptive convolution, an efficient and versatile convolutional module that enhances the model's capability to adaptively represent speech signals. Adaptive convolution performs frame-wise causal dynamic convolution, generating time-varying kernels for each frame by assembling multiple parallel candidate kernels. A lightweight attention mechanism is proposed for adaptive convolution, leveraging both current and historical information to assign adaptive weights to each candidate kernel. This enables the convolution operation to adapt to frame-level speech spectral features, leading to more efficient extraction and reconstruction. We integrate adaptive convolution into various CNN-based models, highlighting its generalizability. Experimental results demonstrate that adaptive convolution significantly improves the performance with negligible increases in computational complexity, especially for lightweight models. Moreover, we present an intuitive analysis revealing a strong correlation between kernel selection and signal characteristics. Furthermore, we propose the adaptive convolutional recurrent network (AdaptCRN), an ultra-lightweight model that incorporates adaptive convolution and an efficient encoder-decoder design, achieving superior performance compared to models with similar or even higher computational costs.

语音增强卷积网络动态卷积

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。