arXiv:2506.12785eess.AScs.SD2025-06被引 2

动态调整卷积核频率响应,显著提升声音事件检测精度。

Frequency Dynamic Convolutions for Sound Event Detection

  • 根据输入信号频率特征动态加权基础卷积核,实现频率自适应建模。
  • 最先进模型在DESED数据集上提升10.98%性能,超越传统2D卷积。
  • 适合处理瞬态、宽频或稳态声音事件,可灵活适配不同场景需求。

基于深度学习的声音事件检测(SED)主要采用卷积循环神经网络(CRNN)和Transformer模型。然而,传统二维卷积假设时间与频率轴的平移不变性,难以捕捉声学信号的频率依赖特性。为此,本文提出频率动态卷积(FDY conv),通过频率特定注意力权重自适应加权多个基础卷积核,构建最优频率响应。实验表明,在DESED数据集上,将FDY conv应用于CRNN可使性能提升7.56%。为进一步突破限制,本文引入扩展系列:稀疏化FDY conv(DFD conv)通过不同膨胀率扩大频率感受野,性能提升9.27%;部分FDY conv(PFD conv)结合标准卷积与频率自适应核,降低计算开销,参数减少54.4%,性能仍提升7.80%;多膨胀FDY conv(MDFD conv)采用多膨胀率核,有效捕捉多样化频率模式,性能提升达10.98%。此外,针对时序平均池化对瞬态事件敏感性不足的问题,提出融合时序注意力(TA)、速度注意力(VA)与平均池化(AP)的TAP-FDY conv(TFD conv),在参数量减少约30.01%的前提下,达到与MDFD conv相当的性能。类别分析显示,不同变体在非平稳、宽谱、准平稳及瞬态事件中各具优势。案例研究验证了各模型在故障识别与弧光检测中的有效性。

原文摘要 · Abstract (English)

Recent research in deep learning-based Sound Event Detection (SED) has primarily focused on Convolutional Recurrent Neural Networks (CRNNs) and Transformer models. However, conventional 2D convolution-based models assume shift invariance along both the temporal and frequency axes, leadin to inconsistencies when dealing with frequency-dependent characteristics of acoustic signals. To address this issue, this study proposes Frequency Dynamic Convolution (FDY conv), which dynamically adjusts convolutional kernels based on the frequency composition of the input signal to enhance SED performance. FDY conv constructs an optimal frequency response by adaptively weighting multiple basis kernels based on frequency-specific attention weights. Experimental results show that applying FDY conv to CRNNs improves performance on the DESED dataset by 7.56% compared to the baseline CRNN. However, FDY conv has limitations in that it combines basis kernels of the same shape across all frequencies, restricting its ability to capture diverse frequency-specific characteristics. Additionally, the $3\times3$ basis kernel size is insufficient to capture a broader frequency range. To overcome these limitations, this study introduces an extended family of FDY conv models. Dilated FDY conv (DFD conv) applies convolutional kernels with various dilation rates to expand the receptive field along the frequency axis and enhance frequency-specific feature representation. Experimental results show that DFD conv improves performance by 9.27% over the baseline. Partial FDY conv (PFD conv) addresses the high computational cost of FDY conv, which results from performing all convolution operations with dynamic kernels. Since FDY conv may introduce unnecessary adaptivity for quasi-stationary sound events, PFD conv integrates standard 2D convolutions with frequency-adaptive kernels to reduce computational complexity while maintaining performance. Experimental results demonstrate that PFD conv improves performance by 7.80% over the baseline while reducing the number of parameters by 54.4% compared to FDY conv. Multi-Dilated FDY conv (MDFD conv) extends DFD conv by addressing its structural limitation of applying the same dilation across all frequencies. By utilizing multiple convolutional kernels with different dilation rates, MDFD conv effectively captures diverse frequency-dependent patterns. Experimental results indicate that MDFD conv achieves the highest performance, improving the baseline CRNN performance by 10.98%. Furthermore, standard FDY conv employs Temporal Average Pooling, which assigns equal weight to all frames along the time axis, limiting its ability to effectively capture transient events. To overcome this, this study proposes TAP-FDY conv (TFD conv), which integrates Temporal Attention Pooling (TA) that focuses on salient features, Velocity Attention Pooling (VA) that emphasizes transient characteristics, and Average Pooling (AP) that captures stationary properties. TAP-FDY conv achieves the same performance as MDFD conv but reduces the number of parameters by approximately 30.01% (12.703M vs. 18.157M), achieving equivalent accuracy with lower computational complexity. Class-wise performance analysis reveals that FDY conv improves detection of non-stationary events, DFD conv is particularly effective for events with broad spectral features, and PFD conv enhances the detection of quasi-stationary events. Additionally, TFD conv (TFD-CRNN) demonstrates strong performance in detecting transient events. In the case studies, PFD conv effectively captures stable signal patterns in tank powertrain fault recognition, DFD conv recognizes wide harmonic spectral patterns on speed-varying motor fault recognition, while TFD conv outperforms other models in detecting transient signals in offshore arc detection. These results suggest that frequency-adaptive convolutions and their extended variants provide a robust alternative to conventional 2D convolutions in deep learning-based audio processing.

声音检测动态卷积频率自适应音频处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。