用音频对齐技术让钢琴家自定义协奏曲伴奏，无需乐谱。
Dense-Sparse Dynamic Time Warping for Customizing Piano Concerto Accompaniments
- 只用音频数据，通过选关键帧对齐来解决音色差异问题。
- 在4个协奏曲片段上表现优于或接近复杂分离方法。
- 适合想个性化伴奏的钢琴演奏者和音乐科技研究者。
本研究探讨钢琴家如何自定义音乐去人声（MMO）协奏曲伴奏以匹配个人演奏风格。无需符号乐谱（常难以数字化获取），我们使用三类音频数据：独奏钢琴录音、仅管弦乐的MMO录音，以及钢琴与管弦乐混合录音（如来自YouTube）。混合录音作为中间参考，仅通过时间尺度调整管弦乐部分以同步用户演奏。主要挑战在于不同录音间因包含不同音乐成分导致的频谱不匹配。为此，我们提出一种改进型动态时间规整（Dense-Sparse DTW），聚焦于对含有显著节拍线索的关键音频帧进行对齐，提升对频谱不匹配的鲁棒性。我们收集并标注了四个钢琴协奏曲乐章的数据，建立了生成与评估定制伴奏的框架。在该基准上，Dense-Sparse DTW性能优于或相当优于基于源分离和频谱减法的复杂方法。
原文摘要 · Abstract (English)
In this study, we explore how pianists can customize Music Minus One (MMO) concerto accompaniments to match their playing style. Bypassing the need for a symbolic score, often not available digitally, we use three types of audio data: solo piano recordings, MMO orchestra-only recordings, and mixed recordings of both piano and orchestra (e.g., from YouTube). The mixed recording serves as an intermediary reference to align the solo and orchestra parts, with only the orchestral part being adjusted through time-scale modification to synchronize with the user's playing. The main challenge with estimating these alignments is the spectral mismatch between recordings containing different musical parts. Motivated by this application scenario, we introduce Dense-Sparse DTW, a variant of Dynamic Time Warping (DTW) that is designed to improve robustness of alignments to spectral mismatch by focusing on aligning a selected subset of audio frames containing prominent timing cues. We collect and annotate data from four piano concerto movements and establish a framework for generating and evaluating customized accompaniment recordings. On this benchmark, we show that Dense-Sparse DTW has better or comparable performance than more complex approaches based on source separation and spectral subtraction techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。