通过交替生成与精修,实现结构连贯的完整歌曲生成。
SongBloom: Coherent Song Generation via Interleaved Autoregressive Sketching and Diffusion Refinement
- 先用自回归粗略生成旋律草图,再用扩散模型逐步细化。
- 在主观和客观评估中均优于现有方法,接近商业平台水平。
- 适合音乐创作、跨模态生成研究者使用。
生成具有连贯结构、和谐乐器与人声配合的音乐仍是重大挑战。现有语言模型和基于扩散的方法难以兼顾全局连贯性与局部保真度,导致输出缺乏乐感或出现逻辑断裂、歌词不匹配等问题。本文提出SongBloom,一种全新的全长歌曲生成框架,采用自回归草图与扩散精修交替的范式。该方法利用自回归扩散模型,融合扩散模型的高保真度与语言模型的可扩展性,逐步将短段音乐草图延展为长篇,并由粗到细完成细节优化。交替生成机制有效整合了语义与声学上下文以指导生成过程。实验表明,SongBloom在主观与客观指标上均优于现有方法,性能接近当前最先进的商用音乐生成平台。音频样例见演示页:https://cypress-yang.github.io/SongBloom_demo。代码与模型权重已开源:https://github.com/Cypress-Yang/SongBloom。
原文摘要 · Abstract (English)
Generating music with coherent structure, harmonious instrumental and vocal elements remains a significant challenge in song generation. Existing language models and diffusion-based methods often struggle to balance global coherence with local fidelity, resulting in outputs that lack musicality or suffer from incoherent progression and mismatched lyrics. This paper introduces $\textbf{SongBloom}$, a novel framework for full-length song generation that leverages an interleaved paradigm of autoregressive sketching and diffusion-based refinement. SongBloom employs an autoregressive diffusion model that combines the high fidelity of diffusion models with the scalability of language models. Specifically, it gradually extends a musical sketch from short to long and refines the details from coarse to fine-grained. The interleaved generation paradigm effectively integrates prior semantic and acoustic context to guide the generation process. Experimental results demonstrate that SongBloom outperforms existing methods across both subjective and objective metrics and achieves performance comparable to the state-of-the-art commercial music generation platforms. Audio samples are available on our demo page: https://cypress-yang.github.io/SongBloom_demo. The code and model weights have been released on https://github.com/Cypress-Yang/SongBloom .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。