音乐生成模型可分别生成并编辑低音、鼓点和其它音轨，实现灵活创作。
MusicGen-Stem: Multi-stem music generation and edition through autoregressive modeling
- 分音轨压缩编码，用自回归模型联合建模三类音轨依赖关系。
- 支持对已有或生成歌曲的任意音轨进行独立编辑，且生成质量高。
- 首个开源多音轨自回归音乐生成模型，适合音乐创作与编辑场景。
现有音乐生成模型通常输出混合音轨（单声道或立体声），本文提出一种包含三类音轨（低音、鼓点、其他）的多音轨生成模型，通过为每类音轨训练专用压缩算法，将音乐转化为并行的令牌流。利用近期在音乐源分离任务上的进展，在大规模数据集上训练多流文本到音乐的语言模型。借助特定条件控制方法，模型可对已有或生成歌曲中的低音、鼓点或其它音轨进行编辑，并支持迭代创作（如在已有鼓点上生成低音）。该方法提升了音乐生成的灵活性，是目前首个公开的高质量多音轨自回归音乐生成模型，具备一致的音轨编辑能力。代码与模型权重将发布，样例可访问 https://simonrouard.github.io/musicgenstem/。
原文摘要 · Abstract (English)
While most music generation models generate a mixture of stems (in mono or stereo), we propose to train a multi-stem generative model with 3 stems (bass, drums and other) that learn the musical dependencies between them. To do so, we train one specialized compression algorithm per stem to tokenize the music into parallel streams of tokens. Then, we leverage recent improvements in the task of music source separation to train a multi-stream text-to-music language model on a large dataset. Finally, thanks to a particular conditioning method, our model is able to edit bass, drums or other stems on existing or generated songs as well as doing iterative composition (e.g. generating bass on top of existing drums). This gives more flexibility in music generation algorithms and it is to the best of our knowledge the first open-source multi-stem autoregressive music generation model that can perform good quality generation and coherent source editing. Code and model weights will be released and samples are available on https://simonrouard.github.io/musicgenstem/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。