开源可复现的长歌生成系统,支持精细风格控制。
Muse: Towards Reproducible Long-Form Song Generation with Fine-Grained Style Control
- 基于Qwen模型扩展离散音频标记,单阶段微调生成歌曲。
- 11.6万首授权合成歌曲训练,音质与风格匹配度优秀。
- 适合音乐生成研究者及需要可控长序列创作的开发者。
近期商业系统如Suno展现出强大的长篇歌曲生成能力,但学术研究因缺乏公开训练数据而难以复现,阻碍了公平比较与进展。为此,我们发布一个完全开源的长篇歌曲生成系统,包含许可的合成数据集、训练与评估流程,以及易部署的Muse歌曲生成模型。数据集包含11.6万首完全授权的合成歌曲,歌词与风格描述由自动生成,音频通过SunoV5合成。Muse通过在Qwen语言模型基础上扩展离散音频标记(MuCodec)进行单阶段监督微调,无需特定任务损失、辅助目标或额外结构。评估显示,尽管数据量和模型规模有限,Muse在音素错误率、文本-音乐风格相似性及音频美学质量方面表现良好,并实现跨不同音乐结构的段级可控生成。所有数据、模型权重及训练评估流程将公开,推动可控长篇歌曲生成研究持续发展。项目仓库地址:https://github.com/yuhui1038/Muse。
原文摘要 · Abstract (English)
Recent commercial systems such as Suno demonstrate strong capabilities in long-form song generation, while academic research remains largely non-reproducible due to the lack of publicly available training data, hindering fair comparison and progress. To this end, we release a fully open-source system for long-form song generation with fine-grained style conditioning, including a licensed synthetic dataset, training and evaluation pipelines, and Muse, an easy-to-deploy song generation model. The dataset consists of 116k fully licensed synthetic songs with automatically generated lyrics and style descriptions paired with audio synthesized by SunoV5. We train Muse via single-stage supervised finetuning of a Qwen-based language model extended with discrete audio tokens using MuCodec, without task-specific losses, auxiliary objectives, or additional architectural components. Our evaluations find that although Muse is trained with a modest data scale and model size, it achieves competitive performance on phoneme error rate, text--music style similarity, and audio aesthetic quality, while enabling controllable segment-level generation across different musical structures. All data, model weights, and training and evaluation pipelines will be publicly released, paving the way for continued progress in controllable long-form song generation research. The project repository is available at https://github.com/yuhui1038/Muse.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。