用文字描述生成可编辑的音乐文件，让非专业人士也能轻松作曲。
Text2midi: Generating Symbolic Music from Captions
- 用大语言模型编码文字提示，驱动自回归解码器生成MIDI序列。
- 人类评估显示生成音乐质量高，且能精准响应调式、和弦等术语。
- 适合音乐创作初学者或需要快速生成旋律的设计师使用。
本文提出Text2midi，一种端到端模型，可将文本描述转换为MIDI文件。该系统利用大规模语言模型（LLMs）处理文字提示，并通过自回归变压器解码器生成符合描述的符号化音乐。我们进行了全面的实验评估，包括自动评测与人工评测，结果表明生成的MIDI文件质量高，且可通过包含调式、和弦、速度等音乐理论术语的文本实现精确控制。代码与演示样本已公开在项目主页（https://github.com/AMAAI-Lab/Text2midi），供用户交互体验。
原文摘要 · Abstract (English)
This paper introduces text2midi, an end-to-end model to generate MIDI files from textual descriptions. Leveraging the growing popularity of multimodal generative approaches, text2midi capitalizes on the extensive availability of textual data and the success of large language models (LLMs). Our end-to-end system harnesses the power of LLMs to generate symbolic music in the form of MIDI files. Specifically, we utilize a pretrained LLM encoder to process captions, which then condition an autoregressive transformer decoder to produce MIDI sequences that accurately reflect the provided descriptions. This intuitive and user-friendly method significantly streamlines the music creation process by allowing users to generate music pieces using text prompts. We conduct comprehensive empirical evaluations, incorporating both automated and human studies, that show our model generates MIDI files of high quality that are indeed controllable by text captions that may include music theory terms such as chords, keys, and tempo. We release the code and music samples on our demo page (https://github.com/AMAAI-Lab/Text2midi) for users to interact with text2midi.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。