构建首个标注完整的演唱动态数据集,提升自动分析精度。
Automatic Estimation of Singing Voice Musical Dynamics
- 用音源分离与对齐技术构建509段带乐谱标注的演唱数据
- 基于频谱特征的模型显示巴克尺度特征更优
- 适合语音分析、音乐信息检索研究者参考
音乐动态是表达性演唱的核心要素。然而,由于缺乏合适的数据集和清晰的评估框架,自动分析演唱动态的研究进展有限。为此,本文提出一种数据集构建方法,利用最先进的音源分离与对齐技术,整理出包含509段标注音乐动态的演唱音频,与163个乐谱文件对齐。乐谱来自广受认可的OpenScore Lieder浪漫主义时期作品语料库,其本身富含表现力标注。基于该数据集,我们训练了一个多头注意力卷积神经网络模型,采用不同窗口大小,并探索了两种感知启发的输入表示:log-Mel谱图与巴克尺度特征。为验证效果,我们联合专业歌手手动构建了25段标注数据用于测试。实验表明,巴克尺度特征在演唱动态预测任务中优于log-Mel特征。相关数据集与代码已公开,供后续研究使用。
原文摘要 · Abstract (English)
Musical dynamics form a core part of expressive singing voice performances. However, automatic analysis of musical dynamics for singing voice has received limited attention partly due to the scarcity of suitable datasets and a lack of clear evaluation frameworks. To address this challenge, we propose a methodology for dataset curation. Employing the proposed methodology, we compile a dataset comprising 509 musical dynamics annotated singing voice performances, aligned with 163 score files, leveraging state-of-the-art source separation and alignment techniques. The scores are sourced from the OpenScore Lieder corpus of romantic-era compositions, widely known for its wealth of expressive annotations. Utilizing the curated dataset, we train a multi-head attention based CNN model with varying window sizes to evaluate the effectiveness of estimating musical dynamics. We explored two distinct perceptually motivated input representations for the model training: log-Mel spectrum and bark-scale based features. For testing, we manually curate another dataset of 25 musical dynamics annotated performances in collaboration with a professional vocalist. We conclude through our experiments that bark-scale based features outperform log-Mel-features for the task of singing voice dynamics prediction. The dataset along with the code is shared publicly for further research on the topic.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。