融合音视频与文本信息，用Mamba模型提升声学场景分类精度
ASCMamba: Multimodal Time-Frequency Mamba for Acoustic Scene Classification
- 采用双路径Mamba结构捕捉音频时频长程依赖
- 通过两阶段伪标签机制提升标注可靠性，性能超越基线6.2%
- 适合关注多模态感知与高效序列建模的研究者
声学场景分类（ASC）是计算听觉中的基础任务，旨在基于声音特征对环境进行识别。在APSIPA ASC 2025大赛中，组织方引入了多模态ASC任务，除音频输入外，还提供录音地点和时间等文本信息。本文提出ASCMamba系统，融合音频与文本实现细粒度声学场景理解。该系统采用DenseEncoder从谱图提取分层频谱特征，再通过双路径Mamba块利用状态空间模型捕捉长期时频依赖。此外，设计两步伪标签机制以生成更可靠的伪标签。实验结果表明，所提方法超越所有参赛团队，在基准上提升6.2%。代码、模型与预训练检查点见https://github.com/S-Orion/ASCMamba.git。
原文摘要 · Abstract (English)
Acoustic Scene Classification (ASC) is a fundamental problem in computational audition, which seeks to classify environments based on the distinctive acoustic features. In the ASC task of the APSIPA ASC 2025 Grand Challenge, the organizers introduce a multimodal ASC task. Unlike traditional ASC systems that rely solely on audio inputs, this challenge provides additional textual information as inputs, including the location where the audio is recorded and the time of recording. In this paper, we present our proposed system for the ASC task in the APSIPA ASC 2025 Grand Challenge. Specifically, we propose a multimodal network, ASCMamba, which integrates audio and textual information for fine-grained acoustic scene understanding and effective multimodal ASC. The proposed ASCMamba employs a DenseEncoder to extract hierarchical spectral features from spectrograms, followed by a dual-path Mamba blocks that capture long-range temporal and frequency dependencies using Mamba-based state space models. In addition, we present a two-step pseudo-labeling mechanism to generate more reliable pseudo-labels. Results show that the proposed system outperforms all the participating teams and achieves a 6.2% improvement over the baseline. Code, model and pre-trained checkpoints are available at https://github.com/S-Orion/ASCMamba.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。