arXiv:2503.06348cs.SDeess.AS2025-03被引 1

用深度学习实现多声部音乐实时伴奏定位,提升演奏跟踪精度。

A Neural Score Follower for Computer Accompaniment of Polyphonic Musical Instruments

  • 设计神经架构学习压缩表示,追踪演奏位置
  • 结合数据增强与启发式规则,提升系统鲁棒性
  • 虽效率高但仍有局限,为后续研究提供方向

针对人声或多声部乐器实时计算机伴奏中的关键问题——乐谱同步定位,现有方法包括字符串动态规划、隐马尔可夫模型和在线时间扭曲。然而深度学习技术尚未被直接应用于该任务。本文提出HeurMiT框架,采用神经网络学习压缩的潜在表示,以在演奏偏离乐谱时仍能精确追踪。同时开发实时MIDI数据增强工具,提升模型鲁棒性,并集成启发式规则,使系统可无缝对接现有转录与伴奏技术。实验表明,尽管计算效率优异,但其内在局限使其难以在真实场景中应用。因此,本文将工作定位为深度学习驱动乐谱跟随的初步探索,旨在揭示未来构建高效鲁棒神经乐谱跟随系统的可行路径。

原文摘要 · Abstract (English)

Real-time computer-based accompaniment for human musical performances entails three critical tasks: identifying what the performer is playing, locating their position within the score, and synchronously playing the accompanying parts. Among these, the second task (score following) has been addressed through methods such as dynamic programming on string sequences, Hidden Markov Models (HMMs), and Online Time Warping (OLTW). Yet, the remarkably successful techniques of Deep Learning (DL) have not been directly applied to this problem. Therefore, we introduce HeurMiT, a novel DL-based score-following framework, utilizing a neural architecture designed to learn compressed latent representations that enables precise performer tracking despite deviations from the score. Parallelly, we implement a real-time MIDI data augmentation toolkit, aimed at enhancing the robustness of these learned representations. Additionally, we integrate the overall system with simple heuristic rules to create a comprehensive framework that can interface seamlessly with existing transcription and accompaniment technologies. However, thorough experimentation reveals that despite its impressive computational efficiency, HeurMiT's underlying limitations prevent it from being practical in real-world score following scenarios. Consequently, we present our work as an introductory exploration into the world of DL-based score followers, while highlighting some promising avenues to encourage future research towards robust, state-of-the-art neural score following systems.

音乐生成深度学习实时同步

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。