让音乐生成可规划:用结构布局控制文本到音乐的创作过程
MusicLayout: Explicit Structural Planning for Controllable Text-to-Music Generation

- 引入音乐布局作为中间表示,显式规划段落与乐器安排
- 在统一自回归框架中先生成布局再生成音频,支持结构修改
- 适合需要精确控制音乐结构的作曲者或交互式创作应用
文本到音乐生成虽快速进步,但现有系统仍主要依赖全局文本提示,导致音乐结构隐含且难以检查、控制或修改。为此,我们提出MusicLayout,一种用于控制文本到音乐生成中音乐结构的显式中间表示。MusicLayout将音乐作品描述为时间对齐的段落、织体、重复、变奏及乐器层级安排,作为文本意图与生成音乐之间的可解释规划层。我们将MusicLayout整合进基于统一自回归框架的文本到音乐系统中,模型先生成MusicLayout表示,再在此基础上预测音频标记,整个过程保持单序列生成。生成的MusicLayout可在音频生成前被检查和修改,实现布局级别的结构控制。通过布局条件生成、布局操控实验和匹配数据消融实验,我们验证了显式布局规划能提升长程结构组织能力,并支持布局级控制。代码已开源:https://github.com/XaryLee/MusicLayout。
原文摘要 · Abstract (English)
Text-to-music generation has advanced rapidly, but current systems still rely primarily on global text prompts, leaving the structural organization of generated music implicit and difficult to inspect, control, or revise before audio generation. To address this issue, we introduce MusicLayout, an explicit intermediate representation for controlling musical structure in text-to-music generation. MusicLayout describes a musical piece as a time-aligned layout of sections, textures, repetitions, variations, and instrument-level arrangements, serving as an interpretable planning layer between textual intent and the generated music. We integrate MusicLayout into a text-to-music framework built on a unified autoregressive formulation, where the model first generates a MusicLayout representation and subsequently predicts audio tokens conditioned on this representation within a single sequence. The resulting MusicLayout can be inspected and modified prior to audio generation, providing a mechanism for layout-level structural control. We evaluate MusicLayout through layout-conditioned generation, layout manipulation experiments, and matched-data ablations, providing evidence that explicit layout planning can improve long-range structural organization and support layout-level control. We have released the implementation as open source on GitHub at https://github.com/XaryLee/MusicLayout.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。