用声音识别吉他扫弦方向和和弦，提升自动转录准确率
Joint Transcription of Acoustic Guitar Strumming Directions and Chords
- 结合真实与合成数据训练深度模型，仅用麦克风音频识别扫弦动作
- 混合数据训练下扫弦检测与和弦识别准确率显著优于基线方法
- 适合音乐信息检索、自动伴奏生成等场景研究者参考
自动吉他扫弦转录是音乐信息检索中的一个未充分研究且具挑战性的任务，尤其在从音频信号中同时提取扫弦方向与和弦进行方面。现有方法常受限于数据集规模。本文通过引入一个新数据集和基于深度学习的转录模型，扩展了多模态吉他扫弦转录方法。我们利用ESP32智能手表运动传感器采集90分钟真实吉他录音，并采用结构化录制协议；同时构建4小时带标签的合成扫弦音频数据集。训练一个卷积循环神经网络（CRNN）模型，仅使用麦克风音频即可检测扫弦事件、分类其方向并识别对应和弦。评估表明，混合合成与真实数据的联合方法在扫弦动作检测和和弦分类上均达到最高准确率，展示了深度学习在鲁棒吉他扫弦转录中的潜力，并为自动节奏吉他分析开辟新路径。
原文摘要 · Abstract (English)
Automatic transcription of guitar strumming is an underrepresented and challenging task in Music Information Retrieval (MIR), particularly for extracting both strumming directions and chord progressions from audio signals. While existing methods show promise, their effectiveness is often hindered by limited datasets. In this work, we extend a multimodal approach to guitar strumming transcription by introducing a novel dataset and a deep learning-based transcription model. We collect 90 min of real-world guitar recordings using an ESP32 smartwatch motion sensor and a structured recording protocol, complemented by a synthetic dataset of 4h of labeled strumming audio. A Convolutional Recurrent Neural Network (CRNN) model is trained to detect strumming events, classify their direction, and identify the corresponding chords using only microphone audio. Our evaluation demonstrates significant improvements over baseline onset detection algorithms, with a hybrid method combining synthetic and real-world data achieving the highest accuracy for both strumming action detection and chord classification. These results highlight the potential of deep learning for robust guitar strumming transcription and open new avenues for automatic rhythm guitar analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。