统一音乐生成与编辑，支持有无MIDI的灵活输入。
P-MUSE: Prompt-MIDI-Optional Model for Unified Instrumental Music Synthesis and Editing
- 分阶段课程学习融合有无MIDI提示的两种模式。
- 同一框架实现音乐生成与局部编辑，提升控制精度。
- 首个涵盖四种乐器的综合性评测基准，适合音频生成研究者。
MIDI-to-Music系统将目标MIDI序列的旋律与节奏转化为音乐片段，并从提示录音中克隆乐器音色。现有方法通常采用两种范式：仅使用提示音频进行条件生成（无需对齐的MIDI），适用于无法获取配对MIDI的情况；或使用配对的提示音频与MIDI进行上下文学习，利用跨模态对齐实现更强的MIDI遵循与音色相似性控制。本文提出P-MUSE，一种统一两种范式的乐器类MIDI-to-Music框架，通过多阶段课程学习支持提示-MIDI-可选输入。此外，P-MUSE通过共享的中间填充公式统一音乐生成与局部编辑。基于理论分析与实证研究，我们提出针对转录到音频系统的相位感知无分类器引导调度原则，以及尾部丢弃策略。最后，为推动该领域研究，我们建立了首个综合性基准，覆盖多种提示模式、生成/编辑任务及四种代表性乐器：钢琴、吉他、贝斯和鼓。演示地址：https://p-muse.github.io/。
原文摘要 · Abstract (English)
MIDI-to-Music system renders the melody and rhythm of a target MIDI sequence into musical segment while cloning instrument timbre from a prompt recording. Existing systems typically adopt one of two distinct paradigms: conditional generation with prompt audio alone, which remains applicable when aligned prompt MIDI is unavailable, and In-Context Learning with paired prompt audio and MIDI, which exploits cross-modal alignment for stronger control on MIDI following and timbre similarity. We introduce P-MUSE, an instrumental MIDI-to-Music framework that unifies both paradigms via a multi-stage Curriculum-Learning supporting prompt-MIDI-optional inputs. P-MUSE further unifies music generation and local editing through a shared fill-in-the-middle formulation. Grounded in theoretical analysis and empirical study, we propose a phase-aware classifier-free guidance scheduling principle for Transcription-to-Audio systems, alongside a Tail-Drop strategy. Finally, to advance research in this field, we establish the first comprehensive benchmark, covering various prompt modes, generation/editing tasks, and four representative instruments: piano, guitar, bass, and drums. Demos are available at https://p-muse.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。