在普通电脑上本地运行大型音乐AI模型，实时生成多乐器乐谱。
Local deployment of large-scale music AI models on commodity hardware
- 将音乐大模型移植到MLC框架，实现跨平台推理。
- 在M3笔记本上每秒生成51音符，86.3%生成可实时播放。
- 适合音乐开发者快速搭建本地化创作工具。
我们提出MIDInfinite，一个基于大尺度生成式AI模型的网页应用，可在普通硬件上本地生成符号化音乐。该演示通过将预训练于Lakh MIDI数据集的前瞻性音乐变压器（Anticipatory Music Transformer）迁移至机器学习编译（MLC）框架实现。迁移后，MLC支持在多种运行时（包括C++、移动端和浏览器）进行推理。我们设想MLC有望弥合日益强大的音乐AI模型与音乐软件开发者更熟悉的技术之间的差距。作为概念验证，我们构建了一个网页应用，用户可在浏览器中从零或根据提示生成无限流的多乐器MIDI。在普通硬件（M3 MacBook Pro）上，演示每秒生成51音符，超过72.9%的生成可实时播放；若预先缓冲2秒，则提升至86.3%。
原文摘要 · Abstract (English)
We present the MIDInfinite, a web application capable of generating symbolic music using a large-scale generative AI model locally on commodity hardware. Creating this demo involved porting the Anticipatory Music Transformer, a large language model (LLM) pre-trained on the Lakh MIDI dataset, to the Machine Learning Compilation (MLC) framework. Once the model is ported, MLC facilitates inference on a variety of runtimes including C++, mobile, and the browser. We envision that MLC has the potential to bridge the gap between the landscape of increasingly capable music AI models and technology more familiar to music software developers. As a proof of concept, we build a web application that allows users to generate endless streams of multi-instrumental MIDI in the browser, either from scratch or conditioned on a prompt. On commodity hardware (an M3 Macbook Pro), our demo can generate 51 notes per second, which is faster than real-time playback for 72.9% of generations, and increases to 86.3% with 2 seconds of upfront buffering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。