对比多种声学特征，发现自监督学习更适配狨猴叫声分析。
On feature representations for marmoset vocal communication analysis
- 比较手工特征、自监督特征与端到端模型在狨猴叫声中的表现。
- 自监督特征和端到端模型在分类任务上优于传统手工特征。
- 信号带宽影响性能，适合语音分析研究者参考。
狨猴（Callithrix jacchus）叫声的声学分析常被用于探究人类语言的进化起源。当前分析多依赖人工或半自动方式，亟需自动化方法。然而现有研究受限于小样本或特定场景，且缺乏对不同任务相关特征的先验认知。本文首次系统探索多种特征表示方法：基于HCTSA的手工特征Catch22、基于人类语音预训练的自监督学习（SSL）特征，以及端到端声学建模。在三个不同的狨猴叫声数据集上验证，结果表明，SSL特征和端到端模型在叫声类型分类、发声者识别和性别识别任务中表现优于Catch22特征。同时，研究揭示了信号带宽对任务性能的重要影响。
原文摘要 · Abstract (English)
The acoustic analysis of marmoset (Callithrix jacchus) vocalizations is often used to understand the evolutionary origins of human language. Currently, the analysis is largely carried out in a manual or semi-manual manner. Thus, there is a need to develop automatic call analysis methods. In that direction, research has been limited to the development of analysis methods with small amounts of data or for specific scenarios. Furthermore, there is lack of prior knowledge about what type of information is relevant for different call analysis tasks. To address these issues, as a first step, this paper explores different feature representation methods, namely, HCTSA-based hand-crafted features Catch22, pre-trained self supervised learning (SSL) based features extracted from neural networks trained on human speech and end-to-end acoustic modeling for call-type classification, caller identification and caller sex identification. Through an investigation on three different marmoset call datasets, we demonstrate that SSL-based feature representations and end-to-end acoustic modeling tend to lead to better systems than Catch22 features for call-type and caller classification. Furthermore, we also highlight the impact of signal bandwidth on the obtained task performances.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。