arXiv:2507.08236cs.SDeess.AS2025-07被引 3

用音谱令牌化实现快速生物声学分类,90分钟内完成206种物种识别。

Distilling Spectrograms into Tokens: Fast and Lightweight Bioacoustic Classification for BirdCLEF+ 2025

  • 将音频转为离散音谱令牌,用词向量模型学习上下文特征。
  • 在90分钟内完成700分钟音频推理,公榜得分0.559,私榜0.520。
  • 适合资源受限场景,如仅用CPU的实时生态监测系统。

BirdCLEF+ 2025挑战要求在严格的90分钟纯CPU推理时限内,对包含206个物种(鸟类、哺乳动物、昆虫、两栖动物)的声音景观录音进行分类,许多先进深度学习方法因此不适用。为应对这一限制,DS@GT团队采用两种策略:首先,通过优化生物声学模型库中的预训练模型以适配CPU推理。使用TFLite后,Perch模型推理速度提升近10倍,约16分钟完成,公榜ROC-AUC达0.729,私榜0.711;模型库中表现最佳的是BirdSetEfficientNetB1,公榜0.810,私榜0.778。其次,提出一种新型轻量级流水线Spectrogram Token Skip-Gram (STSG),将生物声学视为序列建模任务。该方法利用Faiss K-means对梅尔频谱图聚类生成离散音谱令牌,并使用无监督跳字模型(Word2Vec skip-gram)学习高质量上下文嵌入。分类时,5秒窗口内的嵌入取均值后输入线性分类器。预测700分钟测试集仅需约6分钟,公榜得分0.559,私榜0.520,证明静态嵌入的快速令牌化方法在生物声学分类中的可行性。代码见https://github.com/dsgt-arc/birdclef-2025。

原文摘要 · Abstract (English)

The BirdCLEF+ 2025 challenge requires classifying 206 species, including birds, mammals, insects, and amphibians, from soundscape recordings under a strict 90-minute CPU-only inference deadline, making many state-of-the-art deep learning approaches impractical. To address this constraint, the DS@GT BirdCLEF team explored two strategies. First, we establish competitive baselines by optimizing pre-trained models from the Bioacoustics Model Zoo for CPU inference. Using TFLite, we achieved a nearly 10x inference speedup for the Perch model, enabling it to run in approximately 16 minutes and achieve a final ROC-AUC score of 0.729 on the public leaderboard post-competition and 0.711 on the private leaderboard. The best model from the zoo was BirdSetEfficientNetB1, with a public score of 0.810 and a private score of 0.778. Second, we introduce a novel, lightweight pipeline named Spectrogram Token Skip-Gram (STSG) that treats bioacoustics as a sequence modeling task. This method converts audio into discrete "spectrogram tokens" by clustering Mel-spectrograms using Faiss K-means and then learns high-quality contextual embeddings for these tokens in an unsupervised manner with a Word2Vec skip-gram model. For classification, embeddings within a 5-second window are averaged and passed to a linear model. With a projected inference time of 6 minutes for a 700-minute test set, the STSG approach achieved a final ROC-AUC public score of 0.559 and a private score of 0.520, demonstrating the viability of fast tokenization approaches with static embeddings for bioacoustic classification. Supporting code for this paper can be found at https://github.com/dsgt-arc/birdclef-2025.

生物声学轻量化音谱令牌快速推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。