让多轨音乐生成更稳更同步，提升节奏一致性。
SyncTrack: Rhythmic Stability and Synchronization in Multi-Track Music Generation
- 用共享与专属模块协同建模，统一节奏并保留音色差异。
- 新指标显示，生成音乐的节奏稳定性和同步性显著提升。
- 适合音乐生成、音频合成方向研究者参考。
多轨音乐生成因可精准混音和重制而受到广泛关注，但现有模型常忽略节奏稳定性和同步性，导致关注点集中在轨道间差异而非内在特性。本文提出 SyncTrack，一种同步多轨波形音乐生成模型，旨在捕捉多轨音乐的独特属性。SyncTrack 采用新颖架构：包含跨轨共享模块以建立统一节奏，以及轨道专属模块以适应不同音色与音高范围。每个共享模块使用双重跨轨注意力机制同步节奏信息，专属模块则利用可学习乐器先验更好地表征音色等独特特征。此外，我们引入三项新指标评估多轨音乐质量：轨道内节奏稳定性（IRS）、跨轨节拍同步度（CBS）与跨轨节拍离散度（CBD）。实验表明，SyncTrack 显著提升了多轨音乐的节奏一致性与整体质量。
原文摘要 · Abstract (English)
Multi-track music generation has garnered significant research interest due to its precise mixing and remixing capabilities. However, existing models often overlook essential attributes such as rhythmic stability and synchronization, leading to a focus on differences between tracks rather than their inherent properties. In this paper, we introduce SyncTrack, a synchronous multi-track waveform music generation model designed to capture the unique characteristics of multi-track music. SyncTrack features a novel architecture that includes track-shared modules to establish a common rhythm across all tracks and track-specific modules to accommodate diverse timbres and pitch ranges. Each track-shared module employs two cross-track attention mechanisms to synchronize rhythmic information, while each track-specific module utilizes learnable instrument priors to better represent timbre and other unique features. Additionally, we enhance the evaluation of multi-track music quality by introducing rhythmic consistency through three novel metrics: Inner-track Rhythmic Stability (IRS), Cross-track Beat Synchronization (CBS), and Cross-track Beat Dispersion (CBD). Experiments demonstrate that SyncTrack significantly improves the multi-track music quality by enhancing rhythmic consistency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。