统一检测语音与非语音事件，提升复杂音频分析能力
Unified Audio Event Detection
- 构建联合音频事件检测框架，同时识别说话人与声音事件
- 在DESED数据集上相比基线模型性能显著提升
- 适合需要精细音频理解的智能语音系统开发者
声学事件检测（SED）识别声音事件区域，说话人辨识（SD）则分割归属于不同说话人的语音对话。在SED中，所有说话人段落被归为单一语音事件；而在SD中，非语音声音仅被视为背景噪声。因此，两者在包含语音对话与非语音声音的复杂音频场景中均只能提供部分分析。本文提出一种新任务——统一音频事件检测（UAED），实现对非语音事件与基于说话人身份的细粒度语音事件的联合检测。为此，我们设计基于Transformer的T-UAED框架，并构建了源自Librispeech数据集和DESED音效库的UAED数据集。实验表明，该框架有效利用任务间协同作用，显著优于简单合并SED与SD模型输出的基线方法。此外，T-UAED在DESED与CALLHOME数据集上表现接近专用模型，展现出良好泛化能力。
原文摘要 · Abstract (English)
Sound Event Detection (SED) detects regions of sound events, while Speaker Diarization (SD) segments speech conversations attributed to individual speakers. In SED, all speaker segments are classified as a single speech event, while in SD, non-speech sounds are treated merely as background noise. Thus, both tasks provide only partial analysis in complex audio scenarios involving both speech conversation and non-speech sounds. In this paper, we introduce a novel task called Unified Audio Event Detection (UAED) for comprehensive audio analysis. UAED explores the synergy between SED and SD tasks, simultaneously detecting non-speech sound events and fine-grained speech events based on speaker identities. To tackle this task, we propose a Transformer-based UAED (T-UAED) framework and construct the UAED Data derived from the Librispeech dataset and DESED soundbank. Experiments demonstrate that the proposed framework effectively exploits task interactions and substantially outperforms the baseline that simply combines the outputs of SED and SD models. T-UAED also shows its versatility by performing comparably to specialized models for individual SED and SD tasks on DESED and CALLHOME datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。