对比自回归与流匹配在文本生成音乐中的表现差异。
Auto-Regressive vs Flow-Matching: a Comparative Study of Modeling Paradigms for Text-to-Music Generation
- 用相同数据和架构训练模型，只比较两种建模范式。
- 流匹配生成更稳定，自回归在细节控制上更强。
- 适合关注生成稳定性或精细编辑的开发者参考。
近年来，文本到音乐生成取得了显著进展，使模型能够合成高质量的音乐片段、完整作品，甚至响应细粒度控制信号（如和弦进行）。当前最先进的系统在训练数据集、建模范式和架构选择上存在显著差异，这给公平评估模型性能及识别关键设计影响因素带来了挑战。尽管数据与架构等因素重要，本文聚焦于建模范式本身。我们通过系统性实证分析，隔离其影响，揭示相关权衡与涌现行为，为未来文本到音乐生成系统的设计提供指导。具体而言，我们对比了两种最主流的建模范式：自回归解码与条件流匹配。所有模型均从零开始训练，使用相同的训练数据集、配置及相似骨干架构。评估维度包括生成质量、对推理配置的鲁棒性、可扩展性、对文本与时间对齐条件的遵循程度，以及音频修复等编辑能力。该研究揭示了两种范式的独特优势与局限，提供了可操作的洞察，有助于未来架构与训练决策。音频样本可在：https://huggingface.co/spaces/ortal1602/ARvsFM 获取。
原文摘要 · Abstract (English)
Recent progress in text-to-music generation has enabled models to synthesize high-quality musical segments, full compositions, and even respond to fine-grained control signals, e.g. chord progressions. State-of-the-art (SOTA) systems differ significantly in many dimensions, such as training datasets, modeling paradigms, and architectural choices. This diversity complicates efforts to evaluate models fairly and identify which design choices influence performance the most. While factors like data and architecture are important, in this study we focus exclusively on the modeling paradigm. We conduct a systematic empirical analysis to isolate its effects, offering insights into associated trade-offs and emergent behaviors that can guide future text-to-music generation systems. Specifically, we compare the two arguably most common modeling paradigms: auto-regressive decoding and conditional flow-matching. We conduct a controlled comparison by training all models from scratch using identical datasets, training configurations, and similar backbone architectures. Performance is evaluated across multiple axes, including generation quality, robustness to inference configurations, scalability, adherence to both textual and temporally aligned conditioning, and editing capabilities in the form of audio inpainting. This comparative study sheds light on distinct strengths and limitations of each paradigm, providing actionable insights that can inform future architectural and training decisions in the evolving landscape of text-to-music generation. Audio sampled examples are available at: https://huggingface.co/spaces/ortal1602/ARvsFM
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。