一次生成任意组合的多音轨音乐，速度提升50%且质量更高
Stemphonic: All-at-once Flexible Multi-stem Music Generation
- 用共享噪声潜变量并行生成可变数量音轨
- 单次推理完成混音生成，速度比传统方法快25%-50%
- 支持按需控制每轨活动时间，适合音乐创作迭代
音乐音轨生成旨在产出时序同步且独立分离的乐器音频片段，相比传统文本生成音乐模型，能提供更强用户控制力与更贴近音乐人工作流。现有方法要么采用固定架构输出预设音轨集合，要么逐轨生成，导致推理缓慢。本文提出Stemphonic，一种基于扩散/流模型的框架，可在一次推理中生成任意数量的同步音轨，克服速度与灵活性的权衡。训练时将每个音轨视为批次元素，对同步音轨组施加共享噪声潜变量；推理时使用共享初始噪声与音轨特定文本输入，一次性生成多音轨输出。进一步扩展支持条件生成与逐轨活动控制，助力用户迭代编排混音时间层次。在多个开源音轨评估集上验证，Stemphonic在生成质量上表现更优，同时将完整混音生成速度提升25%至50%。演示链接：https://stemphonic-demo.vercel.app
原文摘要 · Abstract (English)
Music stem generation, the task of producing musically-synchronized and isolated instrument audio clips, offers the potential of greater user control and better alignment with musician workflows compared to conventional text-to-music models. Existing stem generation approaches, however, either rely on fixed architectures that output a predefined set of stems in parallel, or generate only one stem at a time, resulting in slow inference despite flexibility in stem combination. We propose Stemphonic, a diffusion-/flow-based framework that overcomes this trade-off and generates a variable set of synchronized stems in one inference pass. During training, we treat each stem as a batch element, group synchronized stems in a batch, and apply a shared noise latent to each group. At inference-time, we use a shared initial noise latent and stem-specific text inputs to generate synchronized multi-stem outputs in one pass. We further expand our approach to enable one-pass conditional multi-stem generation and stem-wise activity controls to empower users to iteratively generate and orchestrate the temporal layering of a mix. We benchmark our results on multiple open-source stem evaluation sets and show that Stemphonic produces higher-quality outputs while accelerating the full mix generation process by 25 to 50%. Demos at: https://stemphonic-demo.vercel.app.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。