用文本查询分离声音，提升重叠声事件检测效果
Exploring Text-Queried Sound Event Detection with Audio Source Separation
- 用文本查询引导音频分离，再分路检测各声音事件
- 在DCASE 2024上比传统方法F1提升7.22%
- 适合需要精准定位复杂声音场景的开发者
在声事件检测（SED）中，重叠声音事件易被背景噪声或其他事件掩盖，导致性能下降。为此，本文提出文本查询声事件检测（TQ-SED）框架：先预训练语言查询音频源分离（LASS）模型，从输入音频中分离出对应不同事件的音轨；再通过多个目标SED分支分别检测各事件。现有先进模型AudioSep因纯卷积结构难以捕捉动态音频信息，为此引入双路径循环神经网络模块，构建AudioSep-DP模型，在DCASE 2024 Task 9语言查询音频分离任务中取得第一名（单模型轨道）。实验表明，TQ-SED显著提升SED性能，相比传统框架F1分数提高7.22%。同时进行了模型复杂度影响的系统性实验。代码与预训练模型已开源。
原文摘要 · Abstract (English)
In sound event detection (SED), overlapping sound events pose a significant challenge, as certain events can be easily masked by background noise or other events, resulting in poor detection performance. To address this issue, we propose the text-queried SED (TQ-SED) framework. Specifically, we first pre-train a language-queried audio source separation (LASS) model to separate the audio tracks corresponding to different events from the input audio. Then, multiple target SED branches are employed to detect individual events. AudioSep is a state-of-the-art LASS model, but has limitations in extracting dynamic audio information because of its pure convolutional structure for separation. To address this, we integrate a dual-path recurrent neural network block into the model. We refer to this structure as AudioSep-DP, which achieves the first place in DCASE 2024 Task 9 on language-queried audio source separation (objective single model track). Experimental results show that TQ-SED can significantly improve the SED performance, with an improvement of 7.22\% on F1 score over the conventional framework. Additionally, we setup comprehensive experiments to explore the impact of model complexity. The source code and pre-trained model are released at https://github.com/apple-yinhan/TQ-SED.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。