用Transformer自动检测音乐演奏错误，准确率提升40个百分点。
Detecting Music Performance Errors with Transformers
- 提出Polytune模型，端到端从音频生成带标注乐谱，隐式对齐音符与演奏
- 合成大规模错误数据集，使14种乐器平均错误检测F1达64.1%
- 支持多乐器同时检测，优于依赖人工规则的旧方法
初学者常难以识别演奏中的具体错误，如音符或节奏错误。现有工具存在两大缺陷：(1) 依赖自动对齐，易受微小偏差影响；(2) 缺乏足够训练数据，导致过度依赖启发式规则。为此，我们提出新型Transformer模型Polytune，直接以音频为输入，输出带标注的乐谱，通过潜在空间表示实现端到端的隐式对齐与比对。为解决数据不足问题，我们开发了一种新颖的数据生成技术，可创建大规模合成音乐错误数据集。实验表明，该方法在14种乐器上平均错误检测F1达到64.1%，相比之前工作提升40个百分点。相较于复用转录模型的方法，本模型支持多乐器检测。代码与数据已开源：https://github.com/ben2002chou/Polytune。
原文摘要 · Abstract (English)
Beginner musicians often struggle to identify specific errors in their performances, such as playing incorrect notes or rhythms. There are two limitations in existing tools for music error detection: (1) Existing approaches rely on automatic alignment; therefore, they are prone to errors caused by small deviations between alignment targets.; (2) There is a lack of sufficient data to train music error detection models, resulting in over-reliance on heuristics. To address (1), we propose a novel transformer model, Polytune, that takes audio inputs and outputs annotated music scores. This model can be trained end-to-end to implicitly align and compare performance audio with music scores through latent space representations. To address (2), we present a novel data generation technique capable of creating large-scale synthetic music error datasets. Our approach achieves a 64.1% average Error Detection F1 score, improving upon prior work by 40 percentage points across 14 instruments. Additionally, compared with existing transcription methods repurposed for music error detection, our model can handle multiple instruments. Our source code and datasets are available at https://github.com/ben2002chou/Polytune.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。