PerTok编码让音乐生成更表达、更高效，还能创作新旋律。
PerTok: Expressive Encoding and Modeling of Symbolic Musical Ideas and Variations
- 用PerTok编码压缩序列长度59%、词表大小95%，保留细微演奏细节
- 两阶段框架：作曲家模型生成创意，演奏者模型添加速度和微时长变化
- 生成音乐既像原风格又新颖，适合音乐人获取灵感
我们提出Cadenza，一种多阶段生成框架，用于预测符号化音乐创意及其富有表现力的变体，以及无条件生成。为此，我们设计了新的MIDI编码方法PerTok（表演标记器），在减少序列长度高达59%、词表大小高达95%的同时，仍能捕捉精细的演奏细节，适用于多声部、单声部和节奏任务。该框架包含两个顺序阶段：1）作曲家模型，基于Transformer的变分自编码器（VAE），采用旋转位置编码（RoPE）并改进自回归解码器以更有效整合输入音乐创意的潜在代码；2）演奏者模型，一个独立训练的双向Transformer编码器，用于预测MIDI序列的速度与微时长。客观与人类评估表明，Cadenza在音乐质量上达到当前无条件生成模型的先进水平，且更具表现力；同时能生成与输入风格相关但具创新性的新音乐创意。本框架旨在伦理地为音乐人提供创作启发。
原文摘要 · Abstract (English)
We introduce Cadenza, a new multi-stage generative framework for predicting expressive variations of symbolic musical ideas as well as unconditional generations. To accomplish this we propose a novel MIDI encoding method, PerTok (Performance Tokenizer) that captures minute expressive details whilst reducing sequence length up to 59% and vocabulary size up to 95% for polyphonic, monophonic and rhythmic tasks. The proposed framework comprises of two sequential stages: 1) Composer and 2) Performer. The Composer model is a transformer-based Variational Autoencoder (VAE), with Rotary Positional Embeddings (RoPE)ROPE and an autoregressive decoder modified to more effectively integrate the latent codes of the input musical idea. The Performer model is a bidirectional transformer encoder that is separately trained to predict velocities and microtimings on MIDI sequences. Objective and human evaluations demonstrate Cadenza's versatile capability in 1) matching other unconditional state-of-the-art symbolic models in musical quality whilst sounding more expressive, and 2) composing new, expressive ideas that are both stylistically related to the input whilst providing novel ideas to the user. Our framework is designed, researched and implemented with the objective of ethically providing inspiration for musicians.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。