用Mamba模型实现更快更准的文本生成音乐，适合资源有限场景。
Exploring State-Space-Model based Language Model in Music Generation
- 用单层向量量化码本表示音乐，简化建模结构。
- 在有限算力下，Mamba模型收敛快且生成音质更接近真实音频。
- 适合追求高效生成的音乐创作与低资源部署场景。
近年来，状态空间模型（SSMs）特别是Mamba的兴起，使其成为跨领域的强大替代或补充模块。本文探索基于Mamba架构的文本到音乐生成潜力。采用残差向量量化（RVQ）离散标记作为建模表示，实证发现单层码本即可捕捉音乐语义信息。受此启发，聚焦于单码本表示，将原本设计为编码器的SiMBA改造为解码器进行序列建模，并与标准Transformer解码器对比。结果表明，在资源受限条件下，SiMBA实现更快收敛，生成结果更接近真实音频。这展示了SSMs在高效且富有表现力的文本到音乐生成中的前景。音频示例已发布至GitHub。
原文摘要 · Abstract (English)
The recent surge in State Space Models (SSMs), particularly the emergence of Mamba, has established them as strong alternatives or complementary modules to Transformers across diverse domains. In this work, we aim to explore the potential of Mamba-based architectures for text-to-music generation. We adopt discrete tokens of Residual Vector Quantization (RVQ) as the modeling representation and empirically find that a single-layer codebook can capture semantic information in music. Motivated by this observation, we focus on modeling a single-codebook representation and adapt SiMBA, originally designed as a Mamba-based encoder, to function as a decoder for sequence modeling. We compare its performance against a standard Transformer-based decoder. Our results suggest that, under limited-resource settings, SiMBA achieves much faster convergence and generates outputs closer to the ground truth. This demonstrates the promise of SSMs for efficient and expressive text-to-music generation. We put audio examples on Github.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。