用原型模型生成伪标签，让无标注音频也参与声音事件检测训练。
Prototype based Masked Audio Model for Self-Supervised Learning of Sound Event Detection
- 基于高斯混合模型构建语义丰富的帧级伪标签
- 在DESED数据集上实现62.5%的PSDS1准确率
- 仅需少量标注数据即可达到顶尖性能，适合数据稀缺场景
声音事件检测（SED）面临的一大挑战是受限于高昂标注成本，导致标注数据稀少。半监督方法依赖高质量标注数据来利用无标注数据，其性能受制于标注数据的质量与数量。本文提出原型掩码音频模型（PMAM），用于SED的自监督表征学习，以更充分地利用无标注数据。具体而言，通过基于高斯混合模型（GMM）的原型分布建模，构建语义丰富的帧级伪标签。这些伪标签用于监督基于Transformer的掩码音频模型训练，采用二元交叉熵损失而非广泛使用的InfoNCE损失，使不同原型的损失贡献独立，适用于多个标签同时作用于未标注帧的真实场景。最后，仅需少量标注数据进行微调，即可获得高性能的SED模型。在与DESED任务一致的测试中，该方法达到62.5%的PSDS1分数，超越现有最先进模型，验证了所提方法的优势。
原文摘要 · Abstract (English)
A significant challenge in sound event detection (SED) is the effective utilization of unlabeled data, given the limited availability of labeled data due to high annotation costs. Semi-supervised algorithms rely on labeled data to learn from unlabeled data, and the performance is constrained by the quality and size of the former. In this paper, we introduce the Prototype based Masked Audio Model~(PMAM) algorithm for self-supervised representation learning in SED, to better exploit unlabeled data. Specifically, semantically rich frame-level pseudo labels are constructed from a Gaussian mixture model (GMM) based prototypical distribution modeling. These pseudo labels supervise the learning of a Transformer-based masked audio model, in which binary cross-entropy loss is employed instead of the widely used InfoNCE loss, to provide independent loss contributions from different prototypes, which is important in real scenarios in which multiple labels may apply to unsupervised data frames. A final stage of fine-tuning with just a small amount of labeled data yields a very high performing SED model. On like-for-like tests using the DESED task, our method achieves a PSDS1 score of 62.5\%, surpassing current state-of-the-art models and demonstrating the superiority of the proposed technique.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。