针对鸟类声音分类难题,改进自编码器模型并提升小样本表现。
Can Masked Autoencoders Also Listen to Birds?
- 重构预训练流程与微调策略,适配鸟类声音数据特征。
- 在BirdSet上达到新最好效果,小样本下性能提升37个百分点。
- 无需微调也能接近全量训练效果,适合资源有限场景。
掩码自编码器(MAE)通过高效的自监督重建任务学习音频分类的丰富语义表征,但在细粒度音频领域泛化能力不足。以鸟类声音分类为例,需区分细微种间差异并应对种内声学变异性,通用音频MAE性能受限。本研究证明,仅靠领域特定预训练数据无法弥合领域差距,必须对整个训练流程进行适配。我们系统地重审并调整了预训练方法、微调策略及冻结特征使用方式,基于与AudioSet规模相当的大型生物声学数据集BirdSet构建Bird-MAE。该模型在BirdSet多标签分类基准上取得新最佳性能。此外,提出参数高效原型探针机制,显著提升冻结特征的利用效率,在低资源条件下逼近全微调性能:原型探针相比线性探针平均精度最高提升37个百分点,并缩小与微调模型的差距。Bird-MAE在新建立的BirdSet少样本基准上展现出强鲁棒性,表明为细粒度音频领域量身定制自监督学习流程的潜力。
原文摘要 · Abstract (English)
Masked Autoencoders (MAEs) learn rich semantic representations in audio classification through an efficient self-supervised reconstruction task. However, general-purpose models fail to generalize well when applied directly to fine-grained audio domains. Specifically, bird-sound classification requires distinguishing subtle inter-species differences and managing high intra-species acoustic variability, revealing the performance limitations of general-domain Audio-MAEs. This work demonstrates that bridging this domain gap domain gap requires full-pipeline adaptation, not just domain-specific pretraining data. We systematically revisit and adapt the pretraining recipe, fine-tuning methods, and frozen feature utilization to bird sounds using BirdSet, a large-scale bioacoustic dataset comparable to AudioSet. Our resulting Bird-MAE achieves new state-of-the-art results in BirdSet's multi-label classification benchmark. Additionally, we introduce the parameter-efficient prototypical probing, enhancing the utility of frozen MAE representations and closely approaching fine-tuning performance in low-resource settings. Bird-MAE's prototypical probes outperform linear probing by up to 37 percentage points in mean average precision and narrow the gap to fine-tuning across BirdSet downstream tasks. Bird-MAE also demonstrates robust few-shot capabilities with prototypical probing in our newly established few-shot benchmark on BirdSet, highlighting the potential of tailored self-supervised learning pipelines for fine-grained audio domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。