arXiv:2412.04217cs.CV2024-12被引 5

解决乐谱与歌词同步转录难题,助力声乐乐谱数字化

Aligned Music Notation and Lyrics Transcription

  • 提出端到端联合建模方法,同时处理乐谱、歌词与对齐关系
  • 语言模型在充足数据下表现最优,对齐准确率显著提升
  • 专设格列高利圣咏数据集,支持精准评估转录与对齐质量

声乐乐谱的数字化面临超越传统光学音乐识别(OMR)和光学字符识别(OCR)的独特挑战,核心在于必须保持乐谱符号与歌词之间的精确对齐。本文首次正式提出并构建了对齐乐谱与歌词转录(AMNLT)任务,旨在通过联合建模音乐符号、歌词及其同步关系,实现声乐乐谱的完整转录。我们系统分析了从分步处理到端到端建模的多种方法,包括直接转录、展开机制与语言建模等。为评估性能,构建了四个格列高利圣咏数据集(含真实与合成数据),并设计专用指标衡量转录与对齐精度。实验表明,端到端方法整体优于启发式方法,尤其在训练数据充足时,语言模型展现出显著优势。本工作建立了首个完整的AMNLT框架,为声乐音乐遗产的数字化提供理论与实践基础。

原文摘要 · Abstract (English)

The digitization of vocal music scores presents unique challenges that go beyond traditional Optical Music Recognition (OMR) and Optical Character Recognition (OCR), as it necessitates preserving the critical alignment between music notation and lyrics. This alignment is essential for proper interpretation and processing in practical applications. This paper introduces and formalizes, for the first time, the Aligned Music Notation and Lyrics Transcription (AMNLT) challenge, which addresses the complete transcription of vocal scores by jointly considering music symbols, lyrics, and their synchronization. We analyze different approaches to address this challenge, ranging from traditional divide-and-conquer methods that handle music and lyrics separately, to novel end-to-end solutions including direct transcription, unfolding mechanisms, and language modeling. To evaluate these methods, we introduce four datasets of Gregorian chants, comprising both real and synthetic sources, along with custom metrics specifically designed to assess both transcription and alignment accuracy. Our experimental results demonstrate that end-to-end approaches generally outperform heuristic methods in the alignment challenge, with language models showing particular promise in scenarios where sufficient training data is available. This work establishes the first comprehensive framework for AMNLT, providing both theoretical foundations and practical solutions for preserving and digitizing vocal music heritage.

音乐转录对齐学习格列高利圣咏

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。