用自监督语音模型识别歌唱中的四种发声模式，准确率超传统方法12%-15%。
voice2mode: Phonation Mode Classification in Singing using Self-Supervised Speech Models
- 从HuBERT和wav2vec2提取分层特征，经全局池化后用轻量分类器判别
- 在763个女高音长音数据上达95.7%准确率，比传统方法提升12-15%
- 底层特征更优，说明声学细节对发声模式识别更重要
本文提出voice2mode，利用大模型自监督语音嵌入对四种歌唱发声模式（气息声、中性声、气流声、压迫声）进行分类。基于公开的女高音数据集（763个持续元音录音，四类标签），从HuBERT及两个wav2vec2变体中提取分层表示，经全局时间池化后，采用SVM、XGBoost等轻量分类器进行判别。实验表明，基础模型特征显著优于传统频谱基线（如频谱图、梅尔频谱图、MFCC），其中早期层的HuBERT嵌入结合SVM实现最高准确率（约95.7%），绝对提升12-15%。进一步分析显示，保留声学/语音细节的底层特征表现优于专用于自动语音识别的顶层特征。
原文摘要 · Abstract (English)
We present voice2mode, a method for classification of four singing phonation modes (breathy, neutral (modal), flow, and pressed) using embeddings extracted from large self-supervised speech models. Prior work on singing phonation has relied on handcrafted signal features or task-specific neural nets; this work evaluates the transferability of speech foundation models to singing phonation classification. voice2mode extracts layer-wise representations from HuBERT and two wav2vec2 variants, applies global temporal pooling, and classifies the pooled embeddings with lightweight classifiers (SVM, XGBoost). Experiments on a publicly available soprano dataset (763 sustained vowel recordings, four labels) show that foundation-model features substantially outperform conventional spectral baselines (spectrogram, mel-spectrogram, MFCC). HuBERT embeddings obtained from early layers yield the best result (~95.7% accuracy with SVM), an absolute improvement of ~12-15% over the best traditional baseline. We also show layer-wise behaviour: lower layers, which retain acoustic/phonetic detail, are more effective than top layers specialized for Automatic Speech Recognition (ASR).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。