让机器学会不同音频轨间的音乐关系，实时生成协调伴奏。
Learning Relationships Between Separate Audio Tracks for Creative Applications
- 用Transformer学习音频轨间的音乐关系，结合符号决策模块
- 在配对音频数据上实现高保真伴奏生成，预测准确率显著提升
- 适合音乐创作、智能编曲等创意应用，无需复杂规则设计
本文是音乐智能体研究项目的第一步，目标是通过训练，使实时生成的音乐输出与现场输入之间达到期望的音乐关系。为此，我们构建了一个分离音轨数据库，并提出一种集成符号决策模块的架构，该模块能从音乐语料中学习并利用音乐关系。我们设计了离线实现方案：以Transformer作为决策模块，基于Wav2Vec 2.0的感知模块，搭配拼接合成作为音频渲染器。定量评估显示，该决策模块能有效复现训练中学习到的关系。实验表明，在给定引导音轨A的情况下，模型可准确预测对应的伴奏音轨B，基于成对音轨数据集（A, B）实现连贯生成。
原文摘要 · Abstract (English)
This paper presents the first step in a research project situated within the field of musical agents. The objective is to achieve, through training, the tuning of the desired musical relationship between a live musical input and a real-time generated musical output, through the curation of a database of separated tracks. We propose an architecture integrating a symbolic decision module capable of learning and exploiting musical relationships from such musical corpus. We detail an offline implementation of this architecture employing Transformers as the decision module, associated with a perception module based on Wav2Vec 2.0, and concatenative synthesis as audio renderer. We present a quantitative evaluation of the decision module's ability to reproduce learned relationships extracted during training. We demonstrate that our decision module can predict a coherent track B when conditioned by its corresponding ''guide'' track A, based on a corpus of paired tracks (A, B).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。