用Mamba提升音调提取效率,结合信心正则化实现半监督学习。
A Mamba-based Network for Semi-supervised Singing Melody Extraction Using Confidence Binary Regularization
- 采用Mamba结构实现线性计算复杂度,提升推理效率。
- 设计基于音符的频谱解码器,更贴近真实音乐表现。
- 引入置信度二值正则化,有效利用未标注数据。
歌声音调提取(SME)是音乐信息检索中的关键任务。现有方法存在三大局限:首先,基于Transformer的模型需二次计算复杂度,导致推理效率低;其次,传统方法依赖频率监督估计基频(f0),忽视了音乐表现本质上以音符为基础;第三,Transformer通常需要大量标注数据才能达到最优性能,但SME任务缺乏充足标注数据。为此,本文提出一种基于Mamba的半监督歌声音调提取网络SpectMamba,采用视觉Mamba实现线性计算复杂度;设计新型音符-基频解码器,使模型更贴合实际音乐表现;并引入置信度二值正则化(CBR)模块,通过最大化正确类别概率来利用未标注数据。在多个公开数据集上的实验验证了该方法的有效性。
原文摘要 · Abstract (English)
Singing melody extraction (SME) is a key task in the field of music information retrieval. However, existing methods are facing several limitations: firstly, prior models use transformers to capture the contextual dependencies, which requires quadratic computation resulting in low efficiency in the inference stage. Secondly, prior works typically rely on frequencysupervised methods to estimate the fundamental frequency (f0), which ignores that the musical performance is actually based on notes. Thirdly, transformers typically require large amounts of labeled data to achieve optimal performances, but the SME task lacks of sufficient annotated data. To address these issues, in this paper, we propose a mamba-based network, called SpectMamba, for semi-supervised singing melody extraction using confidence binary regularization. In particular, we begin by introducing vision mamba to achieve computational linear complexity. Then, we propose a novel note-f0 decoder that allows the model to better mimic the musical performance. Further, to alleviate the scarcity of the labeled data, we introduce a confidence binary regularization (CBR) module to leverage the unlabeled data by maximizing the probability of the correct classes. The proposed method is evaluated on several public datasets and the conducted experiments demonstrate the effectiveness of our proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。