用少量标注数据实现精准鸣禽声波分段,适合标注难的生物声学研究。
Data-Efficient Self-Supervised Algorithms for Fine-Grained Birdsong Analysis
- 三阶段训练:自监督预训练+增强监督微调+半监督优化
- 在极低标注量下对金丝雀鸣声实现高精度分段检测
- 方法可推广至其他鸟类,适用于标注成本高的场景
生物声学、神经科学和语言学研究常以鸣禽鸣声为模型。这需要音频模型对鸣声进行标注与解析,但精确的音节级标注数据稀缺,导致标注成本高。本文提出一种数据高效的鸣禽声波标注器——残差多层感知机循环神经网络,并设计三阶段训练流程:第一阶段利用无标签数据进行自监督学习,探索掩码预测与在线聚类两种主流预训练范式;第二阶段通过有效数据增强进行监督训练,生成个体化的帧级音节检测器;第三阶段采用半监督后训练,利用无标签数据优化个体模型。该方法在极端标注稀缺条件下,成功应用于金丝雀鸣声分析。从信号处理角度看,金丝雀鸣声具有最复杂的时频特征:快速发声、短间隔、高频带扫频及频谱相似音节,需细粒度特征区分。因此,该方法对金丝雀的成功也为其他鸟类提供了可靠基线。案例研究验证其在斑胸草雀鸣声标注中的泛化能力。最后,评估了自监督嵌入在线性探测与无监督分析中的潜力。
原文摘要 · Abstract (English)
Research in bioacoustics, neuroscience, and linguistics often uses birdsong as a proxy to acquire knowledge across diverse areas. This requires audio models to annotate and parse the birdsong. Developing such models requires precise, syllable-level annotated training data. Therefore, automated methods that reduce annotation costs are in demand. This work presents a data-efficient birdsong annotator called Residual Multi-Layer Perceptron Recurrent Neural Network. It then presents a three-stage training pipeline for developing reliable birdsong syllable detectors with minimal annotation. The first stage is self-supervised learning from unlabeled data. Two of the most successful pretraining paradigms are explored, namely, masked prediction and online clustering. The second stage is supervised training with effective data augmentation to produce a robust frame-level syllable detector for each individual. The third stage is a semi-supervised post-training step that refines each individual's model using unlabeled data. The effectiveness of this approach is demonstrated for the Canary song in extreme label-scarcity scenarios. From a signal-processing perspective, the Canary song exhibits one of the most challenging spectro-temporal patterns for algorithmic time-series annotation: rapid vocalizations, brief inter-syllabic intervals, fast and broadband frequency sweeps, and spectrally similar syllables that require fine-grained features to distinguish. Hence, a successful syllable detection algorithm for Canary also establishes a robust baseline for other birds. This methodological generalization is validated in a case study of Bengalese Finch song annotation. Finally, the potential of self-supervised embeddings is assessed for linear probing and unsupervised birdsong analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。