提出首个实时音乐谱面跟踪系统,解决重复与跳转时的定位失准问题。
CODA: Cascaded Online Discontinuity-Aware Alignment for Real-Time Image-Based Score Following

- 分层递进预测:先定位段落,再找小节,最后定音符,保证各层级一致
- 在MSMD钢琴数据集上实现最优实时跟踪准确率与断点恢复能力
- 无需预知重复结构,靠静音触发恢复机制,适合真实演奏场景
从乐谱图像进行实时谱面跟踪仍具挑战,因模型需在严格延迟限制下处理流式音频,并解析高度重复的视觉模式。现有图像方法虽采用多分辨率预测同时输出活跃系统、小节和音符位置,但各层级预测相互独立,导致结果不稳定且增加额外搜索空间。多数方法缺乏对重复、从头开始(D.C.)或尾声跳转(coda)等谱面断点的恢复机制。本文提出CODA,据我们所知首个同时解决上述两个问题的实时谱面跟踪系统。CODA显式利用乐谱的级联结构:先选择活跃系统,再在其内选择活跃小节,最后在选定小节中确定活跃音符,从而强制跨分辨率预测一致性。静音驱动的断点恢复模式可在不依赖重复结构知识的前提下,应对任意谱面断点。在多模态乐谱数据集(MSMD)钢琴基准测试中,CODA在实时吞吐量下达到当前最优的跟踪准确率与断点恢复性能。代码已公开于 https://github.com/ValleyC/CODA。
原文摘要 · Abstract (English)
Real-time score following from sheet images remains chal- lenging because the model must process streaming au- dio while resolving highly repetitive visual patterns un- der strict latency constraints. Recent image-based meth- ods have attempted to use multi-resolution prediction by simultaneously predicting the positions of the active sys- tem, bar, and note. However, their predictions across these different levels of notation are independent, which makes the predictions unstable and introduces unnecessary ex- tra search space for bar- and note-level predictions. Most existing methods also lack mechanisms to recover from score discontinuities, such as repeats, da capo (D.C.), or coda jumps. This paper proposes CODA, to the best of our knowledge, the first real-time score following system that addresses both gaps. CODA explicitly exploits the cascaded structure of music scores: it first selects the ac- tive system, then the active bar within it, and finally the active note within the selected bar. This enforces pre- diction consistency across resolutions. A silence-driven break mode enables recovery from arbitrary score discon- tinuities without requiring knowledge of the repeat struc- ture. Evaluated on the Multimodal Sheet Music Dataset (MSMD) piano benchmarks, CODA achieves state-of-the- art tracking accuracy and discontinuity-recovery perfor- mance under real-time throughput. Code is available at https://github.com/ValleyC/CODA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。